[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:24:59 dist-test-slave-mjjr krb5kdc[8728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:24:59 dist-test-slave-mjjr krb5kdc[8728](info): set up 2 sockets
Aug 09 06:24:59 dist-test-slave-mjjr krb5kdc[8728](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 06:25:01.086529 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.012s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:01 dist-test-slave-mjjr krb5kdc[8728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:01.272135 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45611
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:45611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:01.611397 8750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:01.611831 8750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:01.612074 8750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:01.621387 8750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:01.621558 8750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:01.621670 8750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:01.621773 8750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:01.621865 8750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:01.634141 8750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:45611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:01.635746 8750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:01.637300 8750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:01.655268 8756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:01.655355 8759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:01.655318 8757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:01.657910 8750 server_base.cc:1034] running on GCE node
I20260809 06:25:01.658824 8750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:01.660420 8750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:01.661674 8750 hybrid_clock.cc:648] HybridClock initialized: now 1786256701661634 us; error 83 us; skew 500 ppm
I20260809 06:25:01.662153 8750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:01 dist-test-slave-mjjr krb5kdc[8728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256701, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:01.666548 8750 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:01.668306 8750 webserver.cc:460] Webserver started at http://127.8.131.62:45297/ using document root <none> and password file <none>
I20260809 06:25:01.669027 8750 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:01.669162 8750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:01.669497 8750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:01.673040 8750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "c5a9088aeb20425282953e2ac43cb436"
format_stamp: "Formatted at 2026-08-09 06:25:01 on dist-test-slave-mjjr"
server_key: "ec688eb3e6aff0d6ae5b9a350d661f00"
server_key_iv: "09551b8490615dabf9d9b9c5ff0e2b4a"
server_key_version: "encryptionkey@0"
I20260809 06:25:01.673822 8750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "c5a9088aeb20425282953e2ac43cb436"
format_stamp: "Formatted at 2026-08-09 06:25:01 on dist-test-slave-mjjr"
server_key: "ec688eb3e6aff0d6ae5b9a350d661f00"
server_key_iv: "09551b8490615dabf9d9b9c5ff0e2b4a"
server_key_version: "encryptionkey@0"
I20260809 06:25:01.679325 8750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 06:25:01.683192 8770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:01.684645 8750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:01.684839 8750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "c5a9088aeb20425282953e2ac43cb436"
format_stamp: "Formatted at 2026-08-09 06:25:01 on dist-test-slave-mjjr"
server_key: "ec688eb3e6aff0d6ae5b9a350d661f00"
server_key_iv: "09551b8490615dabf9d9b9c5ff0e2b4a"
server_key_version: "encryptionkey@0"
I20260809 06:25:01.685045 8750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:01.715226 8750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:01.720453 8750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:01.720752 8750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:01.743117 8750 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:45611
I20260809 06:25:01.743158 8843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:45611 every 8 connection(s)
I20260809 06:25:01.744900 8750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:01.749788 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 8750
I20260809 06:25:01.750000 8844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:01.750161 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:01.752972 8716 external_mini_cluster.cc:1428] Setting key c642a499cc85dafc8471b01f274c352a
I20260809 06:25:01.765497 8844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436: Bootstrap starting.
I20260809 06:25:01.769954 8844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:01.771343 8844 log.cc:824] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:01.775360 8844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436: No bootstrap required, opened a new log
I20260809 06:25:01.786098 8844 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a9088aeb20425282953e2ac43cb436" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45611 } }
I20260809 06:25:01.786661 8844 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:01.786801 8844 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a9088aeb20425282953e2ac43cb436, State: Initialized, Role: FOLLOWER
I20260809 06:25:01.787496 8844 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a9088aeb20425282953e2ac43cb436" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45611 } }
I20260809 06:25:01.787824 8844 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:01.788018 8844 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:01.788223 8844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 06:25:01 dist-test-slave-mjjr krb5kdc[8728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:01.791921 8844 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a9088aeb20425282953e2ac43cb436" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45611 } }
I20260809 06:25:01.792543 8844 leader_election.cc:302] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5a9088aeb20425282953e2ac43cb436; no voters:
I20260809 06:25:01.793278 8844 leader_election.cc:288] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:01.793397 8852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:01.794546 8852 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [term 1 LEADER]: Becoming Leader. State: Replica: c5a9088aeb20425282953e2ac43cb436, State: Running, Role: LEADER
I20260809 06:25:01.795183 8852 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a9088aeb20425282953e2ac43cb436" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45611 } }
I20260809 06:25:01.796150 8844 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:01.810655 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:01.758073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45660 (local address 127.8.131.62:45611)
0809 06:25:01.759107 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:01.759128 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:01.761333 (+ 2205us) server_negotiation.cc:407] Connection header received
0809 06:25:01.775479 (+ 14146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:01.775509 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:01.776063 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:01.776530 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:01.780728 (+ 4198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:01.782919 (+ 2191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:01.785349 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:01.786459 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:01.786699 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:01.793805 (+ 7106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:01.793822 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:01.793850 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:01.793912 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:01.798876 (+ 4964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:01.800653 (+ 1777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:01.800667 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:01.800679 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:01.801045 (+ 366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:01.802072 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:01.802081 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:01.802087 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:01.803353 (+ 1266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:01.807807 (+ 4454us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:01.808198 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 06:25:01.808796 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":413,"threads_started":1}
I20260809 06:25:01.812896 8856 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5a9088aeb20425282953e2ac43cb436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a9088aeb20425282953e2ac43cb436" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45611 } } }
I20260809 06:25:01.815574 8856 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:01.813006 8858 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5a9088aeb20425282953e2ac43cb436. Latest consensus state: current_term: 1 leader_uuid: "c5a9088aeb20425282953e2ac43cb436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a9088aeb20425282953e2ac43cb436" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45611 } } }
I20260809 06:25:01.815987 8858 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:01.817198 8865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:01.825721 8865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:01.842537 8865 catalog_manager.cc:1347] Generated new cluster ID: 82947d29413248569ecdbef802e1c6f0
I20260809 06:25:01.842686 8865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:01.867069 8865 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:01.869300 8865 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:01.884815 8865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5a9088aeb20425282953e2ac43cb436: Generated new TSK 0
I20260809 06:25:01.885994 8865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:25:01.953341 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45611
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:02.454726 8883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:02.455106 8883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:02.455299 8883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:02.464998 8883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:02.465190 8883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:02.465412 8883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:02.475713 8883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45611
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:02.477316 8883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:02.478942 8883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:02.493103 8895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:02.493142 8892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:02.493709 8893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:02.494119 8883 server_base.cc:1034] running on GCE node
I20260809 06:25:02.494964 8883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:02.496053 8883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:02.497275 8883 hybrid_clock.cc:648] HybridClock initialized: now 1786256702497212 us; error 79 us; skew 500 ppm
I20260809 06:25:02.497607 8883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:02 dist-test-slave-mjjr krb5kdc[8728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256702, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:02.501212 8883 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:25:02.502841 8883 webserver.cc:460] Webserver started at http://127.8.131.1:46263/ using document root <none> and password file <none>
I20260809 06:25:02.503556 8883 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:02.503717 8883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:02.504067 8883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:02.507102 8883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "bf2d709a48bb4a64a043fbfa6bdfcf62"
format_stamp: "Formatted at 2026-08-09 06:25:02 on dist-test-slave-mjjr"
server_key: "29207e01a3237edcdd8e31864c308d3c"
server_key_iv: "3aa89130dfb739bffce7de85ac682e9e"
server_key_version: "encryptionkey@0"
I20260809 06:25:02.507962 8883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "bf2d709a48bb4a64a043fbfa6bdfcf62"
format_stamp: "Formatted at 2026-08-09 06:25:02 on dist-test-slave-mjjr"
server_key: "29207e01a3237edcdd8e31864c308d3c"
server_key_iv: "3aa89130dfb739bffce7de85ac682e9e"
server_key_version: "encryptionkey@0"
I20260809 06:25:02.513289 8883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 06:25:02.517046 8914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:02.518591 8883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:25:02.518759 8883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "bf2d709a48bb4a64a043fbfa6bdfcf62"
format_stamp: "Formatted at 2026-08-09 06:25:02 on dist-test-slave-mjjr"
server_key: "29207e01a3237edcdd8e31864c308d3c"
server_key_iv: "3aa89130dfb739bffce7de85ac682e9e"
server_key_version: "encryptionkey@0"
I20260809 06:25:02.518965 8883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:02.534968 8883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:02.539402 8883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:02.539765 8883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:02.540784 8883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:02.542176 8883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:02.542280 8883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:02.542438 8883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:02.542512 8883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:02.580552 8883 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:33903
I20260809 06:25:02.580649 9084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:33903 every 8 connection(s)
I20260809 06:25:02.581885 8883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:02.589386 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 8883
I20260809 06:25:02.589639 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:02.590514 8716 external_mini_cluster.cc:1428] Setting key 030a542b890954f6f7a41bac661aa716
Aug 09 06:25:02 dist-test-slave-mjjr krb5kdc[8728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256702, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:02.609844 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:02.585159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:42795 (local address 127.8.131.62:45611)
0809 06:25:02.585745 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:02.585752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:02.586652 (+ 900us) server_negotiation.cc:407] Connection header received
0809 06:25:02.590001 (+ 3349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:02.590010 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:02.590140 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:02.590269 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:02.592652 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:02.593426 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:02.595117 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:02.595552 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:02.595728 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:02.599965 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:02.599972 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:02.599975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:02.600015 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:02.604207 (+ 4192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:02.605708 (+ 1501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:02.605718 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:02.605725 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:02.605907 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:02.606690 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:02.606699 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:02.606705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:02.607173 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:02.607345 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:02.608989 (+ 1644us) server_negotiation.cc:299] Negotiation successful
0809 06:25:02.609298 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":309,"threads_started":1}
I20260809 06:25:02.611181 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:02.585535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45611 (local address 127.8.131.1:42795)
0809 06:25:02.586451 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:02.586516 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:02.589630 (+ 3114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:02.590624 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:02.590640 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:02.591497 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:02.592440 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:02.592471 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:02.593619 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:02.593623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:02.594945 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:02.594961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:02.595834 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:02.595842 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:02.596183 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:02.596979 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:02.597005 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:02.599604 (+ 2599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:02.604471 (+ 4867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:02.604488 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:02.604526 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:02.605473 (+ 947us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:02.606063 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:02.606082 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:02.606088 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:02.606500 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:02.607342 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:02.607353 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:02.608193 (+ 840us) client_negotiation.cc:769] Sending connection context
0809 06:25:02.608641 (+ 448us) client_negotiation.cc:240] Negotiation successful
0809 06:25:02.609383 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":348,"threads_started":1}
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
I20260809 06:25:02.613583 9085 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45611
I20260809 06:25:02.613965 9085 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:02.615144 9085 heartbeater.cc:511] Master 127.8.131.62:45611 requested a full tablet report, sending...
I20260809 06:25:02.617808 8794 ts_manager.cc:194] Registered new tserver with Master: bf2d709a48bb4a64a043fbfa6bdfcf62 (127.8.131.1:33903)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
I20260809 06:25:02.620023 8794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:42795
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:25:02.647451 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45611
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:03.018471 9092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:03.018839 9092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:03.019004 9092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:03.027477 9092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:03.027632 9092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:03.027850 9092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:03.039093 9092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45611
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:03.040582 9092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:03.042022 9092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:03.056752 9100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:03.056902 9104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:03.056731 9102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:03.058113 9092 server_base.cc:1034] running on GCE node
I20260809 06:25:03.058894 9092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:03.060252 9092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:03.061492 9092 hybrid_clock.cc:648] HybridClock initialized: now 1786256703061430 us; error 58 us; skew 500 ppm
I20260809 06:25:03.061895 9092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:03 dist-test-slave-mjjr krb5kdc[8728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:03.065551 9092 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:25:03.067387 9092 webserver.cc:460] Webserver started at http://127.8.131.2:41293/ using document root <none> and password file <none>
I20260809 06:25:03.068389 9092 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:03.068543 9092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:03.068915 9092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:03.072806 9092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "0e0c20758908413a923d7559209d5360"
format_stamp: "Formatted at 2026-08-09 06:25:03 on dist-test-slave-mjjr"
server_key: "93bc69b81bf62b29f76ef560138de3f2"
server_key_iv: "df2e7a726ecef70cb1bbbd801050b877"
server_key_version: "encryptionkey@0"
I20260809 06:25:03.073727 9092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "0e0c20758908413a923d7559209d5360"
format_stamp: "Formatted at 2026-08-09 06:25:03 on dist-test-slave-mjjr"
server_key: "93bc69b81bf62b29f76ef560138de3f2"
server_key_iv: "df2e7a726ecef70cb1bbbd801050b877"
server_key_version: "encryptionkey@0"
I20260809 06:25:03.081003 9092 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260809 06:25:03.086125 9116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:03.088182 9092 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 06:25:03.088398 9092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "0e0c20758908413a923d7559209d5360"
format_stamp: "Formatted at 2026-08-09 06:25:03 on dist-test-slave-mjjr"
server_key: "93bc69b81bf62b29f76ef560138de3f2"
server_key_iv: "df2e7a726ecef70cb1bbbd801050b877"
server_key_version: "encryptionkey@0"
I20260809 06:25:03.088671 9092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:03.142431 9092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:03.147346 9092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:03.147746 9092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:03.148836 9092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:03.150410 9092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:03.150522 9092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:03.150671 9092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:03.150722 9092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:03.190527 9092 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:46045
I20260809 06:25:03.190565 9304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:46045 every 8 connection(s)
I20260809 06:25:03.191905 9092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:03.200033 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9092
I20260809 06:25:03.200250 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:03.200726 8716 external_mini_cluster.cc:1428] Setting key b996439231dc0103dd44df4a39a7c9d8
Aug 09 06:25:03 dist-test-slave-mjjr krb5kdc[8728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:03.217725 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:03.195168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:45207 (local address 127.8.131.62:45611)
0809 06:25:03.195675 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:03.195702 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:03.196742 (+ 1040us) server_negotiation.cc:407] Connection header received
0809 06:25:03.198757 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:03.198762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:03.198855 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:03.198953 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:03.201075 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.202178 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.204212 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.205037 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.205189 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:03.209840 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:03.209849 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:03.209857 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:03.209904 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:03.213510 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.214604 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.214608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.214610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.214757 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.215371 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.215379 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.215385 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.215860 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:03.216035 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:03.217089 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0809 06:25:03.217326 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":230,"threads_started":1}
I20260809 06:25:03.219239 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:03.195788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45611 (local address 127.8.131.2:45207)
0809 06:25:03.196563 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:03.196609 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:03.198488 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:03.199204 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:03.199218 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:03.200025 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:03.200847 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:03.200874 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.202331 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.202337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:03.204003 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:03.204038 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.205474 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.205484 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:03.205796 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:03.206691 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:03.206772 (+ 81us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:03.209568 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:03.213700 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:03.213713 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:03.213736 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:03.214464 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:03.214925 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:03.214933 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:03.214939 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:03.215217 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:03.216049 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:03.216060 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:03.216552 (+ 492us) client_negotiation.cc:769] Sending connection context
0809 06:25:03.216919 (+ 367us) client_negotiation.cc:240] Negotiation successful
0809 06:25:03.217600 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":307,"threads_started":1}
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
I20260809 06:25:03.221145 9306 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45611
I20260809 06:25:03.221534 9306 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:03.222496 9306 heartbeater.cc:511] Master 127.8.131.62:45611 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
I20260809 06:25:03.224321 8794 ts_manager.cc:194] Registered new tserver with Master: 0e0c20758908413a923d7559209d5360 (127.8.131.2:46045)
I20260809 06:25:03.225205 8794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:45207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:25:03.250743 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45611
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:25:03.625067 9085 heartbeater.cc:503] Master 127.8.131.62:45611 was elected leader, sending a full tablet report...
W20260809 06:25:03.630487 9317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:03.630753 9317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:03.630864 9317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:03.640406 9317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:03.640508 9317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:03.640625 9317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:03.651765 9317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45611
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:03.653297 9317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:03.655016 9317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:03.669667 9324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:03.669656 9332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:03.669826 9327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:03.671592 9317 server_base.cc:1034] running on GCE node
I20260809 06:25:03.672312 9317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:03.673336 9317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:03.674557 9317 hybrid_clock.cc:648] HybridClock initialized: now 1786256703674497 us; error 54 us; skew 500 ppm
I20260809 06:25:03.674980 9317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:03 dist-test-slave-mjjr krb5kdc[8728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:03.679498 9317 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:25:03.681422 9317 webserver.cc:460] Webserver started at http://127.8.131.3:40567/ using document root <none> and password file <none>
I20260809 06:25:03.682168 9317 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:03.682332 9317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:03.682684 9317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:03.685846 9317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "a94bd9d28a9748249a572d80fce7b047"
format_stamp: "Formatted at 2026-08-09 06:25:03 on dist-test-slave-mjjr"
server_key: "3c26f04b5b8d7328b1092d748c2eaa24"
server_key_iv: "d7dbcd22f5be73a95d2c4e0e0d45366f"
server_key_version: "encryptionkey@0"
I20260809 06:25:03.686757 9317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "a94bd9d28a9748249a572d80fce7b047"
format_stamp: "Formatted at 2026-08-09 06:25:03 on dist-test-slave-mjjr"
server_key: "3c26f04b5b8d7328b1092d748c2eaa24"
server_key_iv: "d7dbcd22f5be73a95d2c4e0e0d45366f"
server_key_version: "encryptionkey@0"
I20260809 06:25:03.692593 9317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 06:25:03.697140 9344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:03.698742 9317 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 06:25:03.698909 9317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "a94bd9d28a9748249a572d80fce7b047"
format_stamp: "Formatted at 2026-08-09 06:25:03 on dist-test-slave-mjjr"
server_key: "3c26f04b5b8d7328b1092d748c2eaa24"
server_key_iv: "d7dbcd22f5be73a95d2c4e0e0d45366f"
server_key_version: "encryptionkey@0"
I20260809 06:25:03.699122 9317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:03.728677 9317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:03.733676 9317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:03.734028 9317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:03.735136 9317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:03.736699 9317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:03.736816 9317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:03.736982 9317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:03.737061 9317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:03.782785 9317 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:34971
I20260809 06:25:03.782840 9520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:34971 every 8 connection(s)
I20260809 06:25:03.784252 9317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:03.793660 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9317
I20260809 06:25:03.793993 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:03.794960 8716 external_mini_cluster.cc:1428] Setting key 160cda6171a759029b23075ea604800e
Aug 09 06:25:03 dist-test-slave-mjjr krb5kdc[8728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:03.809748 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:03.787449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:48271 (local address 127.8.131.62:45611)
0809 06:25:03.788013 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:03.788018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:03.789774 (+ 1756us) server_negotiation.cc:407] Connection header received
0809 06:25:03.791886 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:03.791892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:03.791990 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:03.792113 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:03.793638 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.794654 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.796882 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.797575 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.797714 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:03.802225 (+ 4511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:03.802230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:03.802233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:03.802268 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:03.805912 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.806910 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.806914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.806917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.807038 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.807495 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.807499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.807501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.807910 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:03.808068 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:03.809078 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0809 06:25:03.809316 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":295,"threads_started":1}
I20260809 06:25:03.810735 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:03.788583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45611 (local address 127.8.131.3:48271)
0809 06:25:03.789607 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:03.789658 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:03.791556 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:03.792356 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:03.792363 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:03.792871 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:03.793450 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:03.793478 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.794823 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.794865 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:03.796733 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:03.796746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.797824 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.797832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:03.798072 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:03.798944 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:03.798982 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:03.801982 (+ 3000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:03.806070 (+ 4088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:03.806082 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:03.806102 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:03.806769 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:03.807154 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:03.807159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:03.807163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:03.807359 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:03.808075 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:03.808085 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:03.808578 (+ 493us) client_negotiation.cc:769] Sending connection context
0809 06:25:03.808970 (+ 392us) client_negotiation.cc:240] Negotiation successful
0809 06:25:03.809506 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":436,"threads_started":1}
I20260809 06:25:03.812559 9521 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45611
I20260809 06:25:03.813019 9521 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:03.814014 9521 heartbeater.cc:511] Master 127.8.131.62:45611 requested a full tablet report, sending...
I20260809 06:25:03.815886 8794 ts_manager.cc:194] Registered new tserver with Master: a94bd9d28a9748249a572d80fce7b047 (127.8.131.3:34971)
I20260809 06:25:03.816696 8794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:48271
I20260809 06:25:03.826627 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 06:25:03 dist-test-slave-mjjr krb5kdc[8728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 06:25:03 dist-test-slave-mjjr krb5kdc[8728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:03.881827 9531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:03.861980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57328 (local address 127.8.131.1:33903)
0809 06:25:03.862882 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:03.862889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:03.862915 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:25:03.862981 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:03.862987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:03.863187 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:03.863365 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:03.865519 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.866853 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.869231 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.870357 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.870509 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:03.875770 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:03.875782 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:03.875797 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:03.875840 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:03.878781 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.879492 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.879503 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.879511 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.879682 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.880113 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.880118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.880122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.880484 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:03.880803 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:03.881149 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 06:25:03.881422 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":565,"threads_started":1}
W20260809 06:25:03.883138 8955 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57328
Aug 09 06:25:03 dist-test-slave-mjjr krb5kdc[8728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 06:25:03 dist-test-slave-mjjr krb5kdc[8728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:03.936141 9531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:03.919876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.8.131.1:33903)
0809 06:25:03.920056 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:03.920061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:03.920603 (+ 542us) server_negotiation.cc:407] Connection header received
0809 06:25:03.920856 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:03.920860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:03.920931 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:03.921060 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:03.922709 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.923449 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.924918 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.925369 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.925556 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:03.930257 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:03.930262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:03.930265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:03.930299 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:03.933352 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.934035 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.934040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.934042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.934183 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.934630 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.934635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.934638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.934940 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:03.935189 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:03.935510 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 06:25:03.935798 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:25:03.937649 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8883
I20260809 06:25:03.951572 9531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:03.938749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.8.131.1:33903)
0809 06:25:03.938975 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:03.938980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:03.939240 (+ 260us) server_negotiation.cc:407] Connection header received
0809 06:25:03.939348 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:03.939351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:03.939403 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:03.939494 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:03.940843 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.941515 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.943113 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:03.943558 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:03.943721 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:03.946049 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:03.946054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:03.946057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:03.946086 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:03.948718 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.949477 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.949481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.949483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.949609 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:03.950186 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:03.950189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:03.950191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:03.950429 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:03.950611 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:03.951127 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 06:25:03.951275 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:25:03.952514 9078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:04.089536 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 8883
I20260809 06:25:04.116799 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9092
Aug 09 06:25:04 dist-test-slave-mjjr krb5kdc[8728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:25:04.139231 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:04.117888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40448 (local address 127.8.131.2:46045)
0809 06:25:04.118863 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:04.118885 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:04.118936 (+ 51us) server_negotiation.cc:407] Connection header received
0809 06:25:04.119105 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:04.119118 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:04.119568 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:04.119914 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:04.121163 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:04.122827 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:04.124696 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:04.126177 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:04.126411 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:04.132569 (+ 6158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:04.132587 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:04.132604 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:04.132650 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:04.135829 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:04.136899 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:04.136906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:04.136908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:04.137032 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:04.137565 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:04.137568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:04.137570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:04.137937 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:04.138170 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:04.138647 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 06:25:04.138870 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":411,"threads_started":1}
I20260809 06:25:04.141194 9295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:04.273593 9306 heartbeater.cc:503] Master 127.8.131.62:45611 was elected leader, sending a full tablet report...
I20260809 06:25:04.277470 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9092
W20260809 06:25:04.301074 8776 connection.cc:535] server connection from 127.8.131.2:45207 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 06:25:04.301401 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9317
Aug 09 06:25:04 dist-test-slave-mjjr krb5kdc[8728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:25:04.322489 9545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:04.302751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47882 (local address 127.8.131.3:34971)
0809 06:25:04.303390 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:04.303397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:04.303447 (+ 50us) server_negotiation.cc:407] Connection header received
0809 06:25:04.303521 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:04.303535 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:04.303841 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:04.304050 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:04.305360 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:04.306366 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:04.307725 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:04.308450 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:04.308526 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:04.313840 (+ 5314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:04.313854 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:04.313872 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:04.313950 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:04.318457 (+ 4507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:04.319549 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:04.319562 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:04.319565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:04.319813 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:04.320375 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:04.320379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:04.320383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:04.320951 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:04.321303 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:04.321679 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 06:25:04.322027 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":289,"threads_started":1}
I20260809 06:25:04.324079 9515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:04.449280 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9317
I20260809 06:25:04.475883 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8750
I20260809 06:25:04.477305 8839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:04.628808 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 8750
2026-08-09T06:25:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4940 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:04 dist-test-slave-mjjr krb5kdc[9555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:04 dist-test-slave-mjjr krb5kdc[9555](info): set up 2 sockets
Aug 09 06:25:04 dist-test-slave-mjjr krb5kdc[9555](info): commencing operation
krb5kdc: starting...
W20260809 06:25:06.060298 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.366s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:06 dist-test-slave-mjjr krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:06.248760 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38203
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:46299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:38203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:06.579798 9574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:06.580121 9574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:06.580325 9574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:06.587983 9574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:06.588135 9574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:06.588235 9574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:06.588352 9574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:06.588445 9574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:06.598964 9574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:46299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:38203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:06.600405 9574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:06.601814 9574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:06.614323 9592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:06.614579 9588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:06.614487 9590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:06.615725 9574 server_base.cc:1034] running on GCE node
I20260809 06:25:06.616559 9574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:06.617981 9574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:06.619225 9574 hybrid_clock.cc:648] HybridClock initialized: now 1786256706619167 us; error 42 us; skew 500 ppm
I20260809 06:25:06.619589 9574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:06 dist-test-slave-mjjr krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256706, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:06.626861 9574 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:06.628495 9574 webserver.cc:460] Webserver started at http://127.8.131.62:34823/ using document root <none> and password file <none>
I20260809 06:25:06.629220 9574 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:06.629371 9574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:06.629741 9574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:06.632808 9574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "63b66a4d7f8940d29f35219d7371bc9f"
format_stamp: "Formatted at 2026-08-09 06:25:06 on dist-test-slave-mjjr"
server_key: "ab4484f16acabec0b67d3cf0d155a9f8"
server_key_iv: "6071583e96880c6578867820a1197fc4"
server_key_version: "encryptionkey@0"
I20260809 06:25:06.633669 9574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "63b66a4d7f8940d29f35219d7371bc9f"
format_stamp: "Formatted at 2026-08-09 06:25:06 on dist-test-slave-mjjr"
server_key: "ab4484f16acabec0b67d3cf0d155a9f8"
server_key_iv: "6071583e96880c6578867820a1197fc4"
server_key_version: "encryptionkey@0"
I20260809 06:25:06.638872 9574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 06:25:06.642346 9603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:06.643625 9574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:25:06.643898 9574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "63b66a4d7f8940d29f35219d7371bc9f"
format_stamp: "Formatted at 2026-08-09 06:25:06 on dist-test-slave-mjjr"
server_key: "ab4484f16acabec0b67d3cf0d155a9f8"
server_key_iv: "6071583e96880c6578867820a1197fc4"
server_key_version: "encryptionkey@0"
I20260809 06:25:06.644112 9574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:06.669209 9574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:06.673151 9574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:06.673442 9574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:06.692415 9574 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:38203
I20260809 06:25:06.692462 9677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:38203 every 8 connection(s)
I20260809 06:25:06.693778 9574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:06.697640 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9574
I20260809 06:25:06.697861 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:06.697904 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:06.698717 8716 external_mini_cluster.cc:1428] Setting key 816eaedb40e094ea9c5716dafb7f83d2
I20260809 06:25:06.710878 9678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f: Bootstrap starting.
Aug 09 06:25:06 dist-test-slave-mjjr krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:06.714390 9678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:06.715814 9678 log.cc:824] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:06.718665 9678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f: No bootstrap required, opened a new log
I20260809 06:25:06.721082 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:06.700956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.8.131.62:38203)
0809 06:25:06.701744 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:06.701755 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:06.701794 (+ 39us) server_negotiation.cc:407] Connection header received
0809 06:25:06.703305 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:06.703325 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:06.703761 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:06.704171 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:06.705683 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:06.706974 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:06.708294 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:06.709008 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:06.709218 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:06.713584 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:06.713592 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:06.713606 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:06.713640 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:06.716975 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:06.717658 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:06.717666 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:06.717672 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:06.717835 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:06.718234 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:06.718239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:06.718243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:06.718808 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:06.719132 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:06.719623 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 06:25:06.720080 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":348,"threads_started":1}
I20260809 06:25:06.726965 9678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b66a4d7f8940d29f35219d7371bc9f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38203 } }
I20260809 06:25:06.727392 9678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:06.727506 9678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b66a4d7f8940d29f35219d7371bc9f, State: Initialized, Role: FOLLOWER
I20260809 06:25:06.728071 9678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b66a4d7f8940d29f35219d7371bc9f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38203 } }
I20260809 06:25:06.728291 9678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:06.728387 9678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:06.728506 9678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:06.730652 9678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b66a4d7f8940d29f35219d7371bc9f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38203 } }
I20260809 06:25:06.731220 9678 leader_election.cc:302] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63b66a4d7f8940d29f35219d7371bc9f; no voters:
I20260809 06:25:06.731837 9678 leader_election.cc:288] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:06.731950 9684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:06.733168 9684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [term 1 LEADER]: Becoming Leader. State: Replica: 63b66a4d7f8940d29f35219d7371bc9f, State: Running, Role: LEADER
I20260809 06:25:06.733700 9684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b66a4d7f8940d29f35219d7371bc9f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38203 } }
I20260809 06:25:06.734139 9678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:06.736215 9685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63b66a4d7f8940d29f35219d7371bc9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b66a4d7f8940d29f35219d7371bc9f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38203 } } }
I20260809 06:25:06.736538 9685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:06.736748 9687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63b66a4d7f8940d29f35219d7371bc9f. Latest consensus state: current_term: 1 leader_uuid: "63b66a4d7f8940d29f35219d7371bc9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b66a4d7f8940d29f35219d7371bc9f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38203 } } }
I20260809 06:25:06.737110 9687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:06.737939 9693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:06.745987 9693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:06.754868 9693 catalog_manager.cc:1347] Generated new cluster ID: 5c0c81b6ea864e388c9d225e4c1c4d63
I20260809 06:25:06.754988 9693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:06.767921 9693 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:06.769286 9693 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:06.774920 9693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63b66a4d7f8940d29f35219d7371bc9f: Generated new TSK 0
I20260809 06:25:06.775863 9693 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:25:06.827107 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38203
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:46299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:07.152626 9723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:07.152915 9723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:07.153012 9723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:07.160074 9723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:07.160187 9723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:07.160305 9723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:07.170022 9723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:46299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38203
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:07.171252 9723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:07.172534 9723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:07.185619 9737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:07.185619 9740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:07.185570 9734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:07.186383 9723 server_base.cc:1034] running on GCE node
I20260809 06:25:07.186950 9723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:07.187875 9723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:07.189072 9723 hybrid_clock.cc:648] HybridClock initialized: now 1786256707189015 us; error 49 us; skew 500 ppm
I20260809 06:25:07.189380 9723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:07 dist-test-slave-mjjr krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256707, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:07.192612 9723 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:25:07.193962 9723 webserver.cc:460] Webserver started at http://127.8.131.1:42787/ using document root <none> and password file <none>
I20260809 06:25:07.194670 9723 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:07.194770 9723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:07.195032 9723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:07.198048 9723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "0b2af47867c748e18a8d1480c7c904a9"
format_stamp: "Formatted at 2026-08-09 06:25:07 on dist-test-slave-mjjr"
server_key: "623bfbb8a6eab05edb5662fa0770cf0c"
server_key_iv: "500d550f99ab35c4d92f4c8fe689b9fd"
server_key_version: "encryptionkey@0"
I20260809 06:25:07.198774 9723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "0b2af47867c748e18a8d1480c7c904a9"
format_stamp: "Formatted at 2026-08-09 06:25:07 on dist-test-slave-mjjr"
server_key: "623bfbb8a6eab05edb5662fa0770cf0c"
server_key_iv: "500d550f99ab35c4d92f4c8fe689b9fd"
server_key_version: "encryptionkey@0"
I20260809 06:25:07.204046 9723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260809 06:25:07.207489 9748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:07.208817 9723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:07.208998 9723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "0b2af47867c748e18a8d1480c7c904a9"
format_stamp: "Formatted at 2026-08-09 06:25:07 on dist-test-slave-mjjr"
server_key: "623bfbb8a6eab05edb5662fa0770cf0c"
server_key_iv: "500d550f99ab35c4d92f4c8fe689b9fd"
server_key_version: "encryptionkey@0"
I20260809 06:25:07.209229 9723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:07.237154 9723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:07.241704 9723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:07.242017 9723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:07.243016 9723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:07.244371 9723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:07.244474 9723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:07.244619 9723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:07.244692 9723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:07.278291 9723 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:41583
I20260809 06:25:07.278388 9915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:41583 every 8 connection(s)
I20260809 06:25:07.279563 9723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:07.286222 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9723
I20260809 06:25:07.286474 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:07.287077 8716 external_mini_cluster.cc:1428] Setting key 4811d1928cc09a74f17c48d02d5ae526
Aug 09 06:25:07 dist-test-slave-mjjr krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256707, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:07.300057 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:07.282125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:44525 (local address 127.8.131.62:38203)
0809 06:25:07.282561 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:07.282566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:07.283549 (+ 983us) server_negotiation.cc:407] Connection header received
0809 06:25:07.285424 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:07.285430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:07.285531 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:07.285657 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:07.287252 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:07.288184 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:07.289873 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:07.290282 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:07.290429 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:07.294204 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:07.294209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:07.294212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:07.294244 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:07.296923 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:07.297671 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:07.297678 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:07.297682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:07.297809 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:07.298242 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:07.298249 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:07.298252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:07.298522 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:07.298679 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:07.299407 (+ 728us) server_negotiation.cc:299] Negotiation successful
0809 06:25:07.299663 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":231,"threads_started":1}
I20260809 06:25:07.301138 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:07.282702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38203 (local address 127.8.131.1:44525)
0809 06:25:07.283387 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:07.283431 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:07.285166 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:07.285891 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:07.285899 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:07.286474 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:07.287067 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:07.287084 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:07.288344 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:07.288349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:07.289710 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:07.289723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:07.290527 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:07.290537 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:07.290767 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:07.291457 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:07.291479 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:07.293943 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:07.297087 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:07.297096 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:07.297112 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:07.297549 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:07.297923 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:07.297930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:07.297934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:07.298125 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:07.298654 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:07.298661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:07.299016 (+ 355us) client_negotiation.cc:769] Sending connection context
0809 06:25:07.299284 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 06:25:07.299856 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":278,"threads_started":1}
I20260809 06:25:07.303150 9916 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38203
I20260809 06:25:07.303462 9916 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:07.304229 9916 heartbeater.cc:511] Master 127.8.131.62:38203 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
I20260809 06:25:07.306375 9627 ts_manager.cc:194] Registered new tserver with Master: 0b2af47867c748e18a8d1480c7c904a9 (127.8.131.1:41583)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
I20260809 06:25:07.308038 9627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:44525
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:25:07.331518 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38203
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:46299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:07.663410 9931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:07.663815 9931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:07.663947 9931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:07.671350 9931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:07.671463 9931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:07.671583 9931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:07.682227 9931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:46299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38203
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:07.683506 9931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:07.685016 9931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:07.699080 9938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:07.699057 9940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:07.699159 9944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:07.700337 9931 server_base.cc:1034] running on GCE node
I20260809 06:25:07.701121 9931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:07.702281 9931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:07.703542 9931 hybrid_clock.cc:648] HybridClock initialized: now 1786256707703489 us; error 33 us; skew 500 ppm
I20260809 06:25:07.703994 9931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:07 dist-test-slave-mjjr krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256707, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:07.708060 9931 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:25:07.711433 9931 webserver.cc:460] Webserver started at http://127.8.131.2:35307/ using document root <none> and password file <none>
I20260809 06:25:07.712383 9931 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:07.712527 9931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:07.712867 9931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:07.715940 9931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "ddf626f484ab475d8c43fd6274b7fc83"
format_stamp: "Formatted at 2026-08-09 06:25:07 on dist-test-slave-mjjr"
server_key: "f0410bd6c8b80f620dd96c8b3db674d1"
server_key_iv: "96402880756bbfa1b97838bac135bec6"
server_key_version: "encryptionkey@0"
I20260809 06:25:07.716724 9931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "ddf626f484ab475d8c43fd6274b7fc83"
format_stamp: "Formatted at 2026-08-09 06:25:07 on dist-test-slave-mjjr"
server_key: "f0410bd6c8b80f620dd96c8b3db674d1"
server_key_iv: "96402880756bbfa1b97838bac135bec6"
server_key_version: "encryptionkey@0"
I20260809 06:25:07.721930 9931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 06:25:07.725449 9956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:07.726773 9931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:25:07.726955 9931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "ddf626f484ab475d8c43fd6274b7fc83"
format_stamp: "Formatted at 2026-08-09 06:25:07 on dist-test-slave-mjjr"
server_key: "f0410bd6c8b80f620dd96c8b3db674d1"
server_key_iv: "96402880756bbfa1b97838bac135bec6"
server_key_version: "encryptionkey@0"
I20260809 06:25:07.727154 9931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:07.767962 9931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:07.772470 9931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:07.772845 9931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:07.773936 9931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:07.775432 9931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:07.775549 9931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:07.775744 9931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:07.775851 9931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:07.811573 9931 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:43373
I20260809 06:25:07.811635 10123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:43373 every 8 connection(s)
I20260809 06:25:07.813110 9931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:07.822472 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9931
I20260809 06:25:07.822693 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:07.823165 8716 external_mini_cluster.cc:1428] Setting key da6b21fce292254827f346a1179c5efb
Aug 09 06:25:07 dist-test-slave-mjjr krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256707, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:07.835143 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:07.815969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:58863 (local address 127.8.131.62:38203)
0809 06:25:07.816443 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:07.816449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:07.817471 (+ 1022us) server_negotiation.cc:407] Connection header received
0809 06:25:07.819216 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:07.819220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:07.819292 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:07.819392 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:07.820952 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:07.821672 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:07.823300 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:07.823846 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:07.824043 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:07.828090 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:07.828100 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:07.828108 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:07.828153 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:07.831793 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:07.832776 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:07.832780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:07.832782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:07.832909 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:07.833404 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:07.833407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:07.833410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:07.833663 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:07.833798 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:07.834685 (+ 887us) server_negotiation.cc:299] Negotiation successful
0809 06:25:07.834874 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":222,"threads_started":1}
I20260809 06:25:07.836793 10130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:07.816567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38203 (local address 127.8.131.2:58863)
0809 06:25:07.817290 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:07.817339 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:07.818968 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:07.819599 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:07.819608 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:07.820198 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:07.820779 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:07.820796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:07.821802 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:07.821805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:07.823155 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:07.823167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:07.824139 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:07.824149 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:07.824429 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:07.825167 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:07.825203 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:07.827856 (+ 2653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:07.831921 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:07.831933 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:07.831951 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:07.832630 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:07.833030 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:07.833038 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:07.833045 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:07.833291 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:07.833797 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:07.833809 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:07.834286 (+ 477us) client_negotiation.cc:769] Sending connection context
0809 06:25:07.834596 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 06:25:07.835206 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":294,"threads_started":1}
I20260809 06:25:07.838307 10124 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38203
I20260809 06:25:07.838630 10124 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:07.839390 10124 heartbeater.cc:511] Master 127.8.131.62:38203 requested a full tablet report, sending...
I20260809 06:25:07.841091 9627 ts_manager.cc:194] Registered new tserver with Master: ddf626f484ab475d8c43fd6274b7fc83 (127.8.131.2:43373)
I20260809 06:25:07.841938 9627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:58863
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:25:07.869354 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38203
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:46299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:08.196883 10133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:08.197187 10133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:08.197372 10133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:08.204327 10133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:08.204470 10133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:08.204659 10133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:08.214425 10133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:46299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38203
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:08.215734 10133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:08.217046 10133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:08.230010 10143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:08.230059 10141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:08.230086 10146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:08.231426 10133 server_base.cc:1034] running on GCE node
I20260809 06:25:08.232084 10133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:08.232982 10133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:08.234185 10133 hybrid_clock.cc:648] HybridClock initialized: now 1786256708234148 us; error 32 us; skew 500 ppm
I20260809 06:25:08.234525 10133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:08 dist-test-slave-mjjr krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:08.237798 10133 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:25:08.239215 10133 webserver.cc:460] Webserver started at http://127.8.131.3:46249/ using document root <none> and password file <none>
I20260809 06:25:08.240010 10133 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:08.240151 10133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:08.240504 10133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:08.243440 10133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "468600403a7343d1b03daa38ca9b0166"
format_stamp: "Formatted at 2026-08-09 06:25:08 on dist-test-slave-mjjr"
server_key: "f8aa34b666933845ac079e45b6a17845"
server_key_iv: "ca4728b7a8ad149dc58a58eee463b3be"
server_key_version: "encryptionkey@0"
I20260809 06:25:08.244540 10133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "468600403a7343d1b03daa38ca9b0166"
format_stamp: "Formatted at 2026-08-09 06:25:08 on dist-test-slave-mjjr"
server_key: "f8aa34b666933845ac079e45b6a17845"
server_key_iv: "ca4728b7a8ad149dc58a58eee463b3be"
server_key_version: "encryptionkey@0"
I20260809 06:25:08.250030 10133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:08.253407 10157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:08.254650 10133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:25:08.254817 10133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "468600403a7343d1b03daa38ca9b0166"
format_stamp: "Formatted at 2026-08-09 06:25:08 on dist-test-slave-mjjr"
server_key: "f8aa34b666933845ac079e45b6a17845"
server_key_iv: "ca4728b7a8ad149dc58a58eee463b3be"
server_key_version: "encryptionkey@0"
I20260809 06:25:08.255007 10133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:08.269120 10133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:08.273434 10133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:08.273739 10133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:08.274647 10133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:08.275971 10133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:08.276054 10133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:08.276225 10133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:08.276283 10133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:08.310617 10133 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:44101
I20260809 06:25:08.310662 10328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:44101 every 8 connection(s)
I20260809 06:25:08.311817 9916 heartbeater.cc:503] Master 127.8.131.62:38203 was elected leader, sending a full tablet report...
I20260809 06:25:08.312134 10133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:08.319011 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 10133
I20260809 06:25:08.319216 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:08.319772 8716 external_mini_cluster.cc:1428] Setting key d2801e9c4cb9126f862db46f9c8b526f
Aug 09 06:25:08 dist-test-slave-mjjr krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:08.334082 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.315084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:46531 (local address 127.8.131.62:38203)
0809 06:25:08.315266 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.315271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.316445 (+ 1174us) server_negotiation.cc:407] Connection header received
0809 06:25:08.318160 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.318164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.318229 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.318317 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:08.320329 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.321433 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.323297 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.323761 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.323863 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.327480 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:08.327485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:08.327488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:08.327520 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:08.330300 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.331277 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.331281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.331284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.331444 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.331988 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.331993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.332002 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.332308 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:08.332445 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.333375 (+ 930us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.333646 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:25:08.335186 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.315525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38203 (local address 127.8.131.3:46531)
0809 06:25:08.316290 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:08.316337 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:08.317925 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:08.318522 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:08.318529 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:08.319297 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:08.320136 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.320163 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.321597 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.321601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.323168 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.323181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.323948 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.323953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.324152 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.324748 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:08.324769 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:08.327298 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:08.330482 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:08.330490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:08.330504 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:08.331104 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:08.331576 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:08.331579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:08.331582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:08.331849 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:08.332511 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:08.332518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:08.332897 (+ 379us) client_negotiation.cc:769] Sending connection context
0809 06:25:08.333239 (+ 342us) client_negotiation.cc:240] Negotiation successful
0809 06:25:08.333844 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":296,"threads_started":1}
I20260809 06:25:08.336884 10329 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38203
I20260809 06:25:08.337225 10329 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:08.337992 10329 heartbeater.cc:511] Master 127.8.131.62:38203 requested a full tablet report, sending...
I20260809 06:25:08.339437 9627 ts_manager.cc:194] Registered new tserver with Master: 468600403a7343d1b03daa38ca9b0166 (127.8.131.3:44101)
I20260809 06:25:08.340287 9627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:46531
I20260809 06:25:08.347731 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:25:08.366364 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.354004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56466 (local address 127.8.131.62:38203)
0809 06:25:08.354207 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.354214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.354679 (+ 465us) server_negotiation.cc:407] Connection header received
0809 06:25:08.354827 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.354831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.354897 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.354993 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:08.356495 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.357221 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.358379 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.358882 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.358975 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.361472 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:08.361477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:08.361480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:08.361508 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:08.364052 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.364666 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.364670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.364672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.364777 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.365109 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.365112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.365114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.365402 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:08.365581 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.365831 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.366003 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:25:08.372344 9627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:25:08.378976 9627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:08.404062 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.390853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41583 (local address 127.0.0.1:39934)
0809 06:25:08.391701 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:08.391765 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:08.391950 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:08.393531 (+ 1581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:08.393542 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:08.393586 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:08.394168 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.394184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.397073 (+ 2889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.397082 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.400287 (+ 3205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.400310 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.402911 (+ 2601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.402920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.403165 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.403222 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 06:25:08.403345 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 06:25:08.403553 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"spinlock_wait_cycles":10624,"thread_start_us":987,"threads_started":1}
I20260809 06:25:08.404857 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.392002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.8.131.1:41583)
0809 06:25:08.392741 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.392753 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.392792 (+ 39us) server_negotiation.cc:407] Connection header received
0809 06:25:08.392911 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.392921 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.393241 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.394438 (+ 1197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:08.394722 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.396858 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.400570 (+ 3712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.402622 (+ 2052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.403996 (+ 1374us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.404251 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.404341 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.404513 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":396,"threads_started":1}
I20260809 06:25:08.408428 9821 tablet_service.cc:1467] Processing CreateTablet for tablet b45ee26dc47b41499826845380018b24 (DEFAULT_TABLE table=test-table [id=b6d43c415a79416185a90048735c55bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:08.410113 9821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b45ee26dc47b41499826845380018b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:08.411399 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.391397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.8.131.3:44101)
0809 06:25:08.403347 (+ 11950us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.403357 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.403390 (+ 33us) server_negotiation.cc:407] Connection header received
0809 06:25:08.403505 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.403513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.403776 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.405050 (+ 1274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:08.405299 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.406799 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.409076 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.410276 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.410435 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.410657 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.410902 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.411048 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11771,"thread_start_us":11607,"threads_started":1}
I20260809 06:25:08.411393 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.390926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:44101 (local address 127.0.0.1:57672)
0809 06:25:08.391701 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:08.391765 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:08.391950 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:08.404010 (+ 12060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:08.404017 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:08.404055 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:08.404834 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.404845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.406954 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.406960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.408878 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.408893 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.410497 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.410503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.410683 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.410725 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 06:25:08.410848 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 06:25:08.410976 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":46,"spinlock_wait_cycles":4736,"thread_start_us":331,"threads_started":1}
I20260809 06:25:08.412981 10355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.392193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:43373 (local address 127.0.0.1:49126)
0809 06:25:08.403661 (+ 11468us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:08.403681 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:08.404660 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:08.405997 (+ 1337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:08.406002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:08.406028 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:08.406361 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.406369 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.408173 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.408179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.410286 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.410301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.412143 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.412150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.412340 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.412372 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 06:25:08.412468 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 06:25:08.412600 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11313,"mutex_wait_us":1,"thread_start_us":11266,"threads_started":1}
I20260809 06:25:08.413100 10358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.392530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.8.131.2:43373)
0809 06:25:08.403437 (+ 10907us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.403447 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.403862 (+ 415us) server_negotiation.cc:407] Connection header received
0809 06:25:08.404887 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.404896 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.405815 (+ 919us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.406008 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:08.406534 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.408020 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.410496 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.411920 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.412074 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.412280 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.412544 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.412707 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10697,"thread_start_us":11102,"threads_started":1}
I20260809 06:25:08.415333 10230 tablet_service.cc:1467] Processing CreateTablet for tablet b45ee26dc47b41499826845380018b24 (DEFAULT_TABLE table=test-table [id=b6d43c415a79416185a90048735c55bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:08.416822 10230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b45ee26dc47b41499826845380018b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:08.416462 10037 tablet_service.cc:1467] Processing CreateTablet for tablet b45ee26dc47b41499826845380018b24 (DEFAULT_TABLE table=test-table [id=b6d43c415a79416185a90048735c55bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:08.417725 10037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b45ee26dc47b41499826845380018b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:08.424516 10359 tablet_bootstrap.cc:492] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9: Bootstrap starting.
I20260809 06:25:08.427368 10359 tablet_bootstrap.cc:654] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:08.428491 10359 log.cc:824] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:08.431684 10359 tablet_bootstrap.cc:492] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9: No bootstrap required, opened a new log
I20260809 06:25:08.431993 10359 ts_tablet_manager.cc:1397] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 06:25:08.432564 10363 tablet_bootstrap.cc:492] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166: Bootstrap starting.
I20260809 06:25:08.434047 10364 tablet_bootstrap.cc:492] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83: Bootstrap starting.
I20260809 06:25:08.436517 10363 tablet_bootstrap.cc:654] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:08.436949 10364 tablet_bootstrap.cc:654] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:08.437747 10363 log.cc:824] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:08.437901 10364 log.cc:824] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:08.440927 10364 tablet_bootstrap.cc:492] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83: No bootstrap required, opened a new log
I20260809 06:25:08.440932 10363 tablet_bootstrap.cc:492] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166: No bootstrap required, opened a new log
I20260809 06:25:08.441197 10364 ts_tablet_manager.cc:1397] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 06:25:08.441201 10363 ts_tablet_manager.cc:1397] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 06:25:08.442070 10359 raft_consensus.cc:348] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } }
I20260809 06:25:08.442516 10359 raft_consensus.cc:374] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:08.442658 10359 raft_consensus.cc:729] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2af47867c748e18a8d1480c7c904a9, State: Initialized, Role: FOLLOWER
I20260809 06:25:08.443257 10359 consensus_queue.cc:260] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } }
I20260809 06:25:08.445030 10359 ts_tablet_manager.cc:1428] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260809 06:25:08.452173 10363 raft_consensus.cc:348] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } }
I20260809 06:25:08.452173 10364 raft_consensus.cc:348] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } }
I20260809 06:25:08.452617 10364 raft_consensus.cc:374] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:08.452617 10363 raft_consensus.cc:374] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:08.452728 10364 raft_consensus.cc:729] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf626f484ab475d8c43fd6274b7fc83, State: Initialized, Role: FOLLOWER
I20260809 06:25:08.452728 10363 raft_consensus.cc:729] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468600403a7343d1b03daa38ca9b0166, State: Initialized, Role: FOLLOWER
I20260809 06:25:08.453341 10363 consensus_queue.cc:260] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } }
I20260809 06:25:08.453356 10364 consensus_queue.cc:260] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } }
I20260809 06:25:08.454478 10124 heartbeater.cc:503] Master 127.8.131.62:38203 was elected leader, sending a full tablet report...
I20260809 06:25:08.455375 10364 ts_tablet_manager.cc:1428] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83: Time spent starting tablet: real 0.014s user 0.007s sys 0.008s
I20260809 06:25:08.456465 10329 heartbeater.cc:503] Master 127.8.131.62:38203 was elected leader, sending a full tablet report...
I20260809 06:25:08.457083 10363 ts_tablet_manager.cc:1428] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260809 06:25:08.532061 9917 tablet.cc:2369] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:25:08.564248 10332 tablet.cc:2369] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:25:08.565455 10126 tablet.cc:2369] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:25:08.677685 10367 raft_consensus.cc:482] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:08.677987 10367 raft_consensus.cc:504] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } }
I20260809 06:25:08.679838 10367 leader_election.cc:288] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 468600403a7343d1b03daa38ca9b0166 (127.8.131.3:44101), ddf626f484ab475d8c43fd6274b7fc83 (127.8.131.2:43373)
I20260809 06:25:08.689337 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.680480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:32939 (local address 127.8.131.3:44101)
0809 06:25:08.680789 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.680795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.681376 (+ 581us) server_negotiation.cc:407] Connection header received
0809 06:25:08.681582 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.681588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.681667 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.681771 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:08.682708 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.683868 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.686907 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.688197 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.688340 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.688443 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.688934 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.689067 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260809 06:25:08.689350 10358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.680492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:44437 (local address 127.8.131.2:43373)
0809 06:25:08.680727 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.680736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.681399 (+ 663us) server_negotiation.cc:407] Connection header received
0809 06:25:08.681582 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.681588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.681667 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.681771 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:08.682685 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.683865 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.686907 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.688211 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.688352 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.688437 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.688934 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.689067 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:25:08.689612 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.680318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:43373 (local address 127.8.131.1:44437)
0809 06:25:08.681276 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:08.681292 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:08.681419 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:08.681807 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:08.681812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:08.681859 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:08.682448 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.682488 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.684017 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.684021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.686728 (+ 2707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.686749 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.688502 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.688507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.688725 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.688793 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 06:25:08.688892 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 06:25:08.689112 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":316,"spinlock_wait_cycles":18816,"thread_start_us":684,"threads_started":1}
I20260809 06:25:08.689932 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.680202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:44101 (local address 127.8.131.1:32939)
0809 06:25:08.681221 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:08.681241 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:08.681426 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:08.681807 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:08.681812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:08.681860 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:08.682570 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.682584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.684015 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.684020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.686748 (+ 2728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:08.686758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.688472 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.688484 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:08.688729 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.688797 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 06:25:08.688892 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 06:25:08.689112 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":300,"spinlock_wait_cycles":57600,"thread_start_us":513,"threads_started":1}
I20260809 06:25:08.690726 10064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b45ee26dc47b41499826845380018b24" candidate_uuid: "0b2af47867c748e18a8d1480c7c904a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf626f484ab475d8c43fd6274b7fc83" is_pre_election: true
I20260809 06:25:08.690886 10265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b45ee26dc47b41499826845380018b24" candidate_uuid: "0b2af47867c748e18a8d1480c7c904a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468600403a7343d1b03daa38ca9b0166" is_pre_election: true
I20260809 06:25:08.691287 10064 raft_consensus.cc:2454] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2af47867c748e18a8d1480c7c904a9 in term 0.
I20260809 06:25:08.691413 10265 raft_consensus.cc:2454] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2af47867c748e18a8d1480c7c904a9 in term 0.
I20260809 06:25:08.692001 9750 leader_election.cc:302] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2af47867c748e18a8d1480c7c904a9, ddf626f484ab475d8c43fd6274b7fc83; no voters:
I20260809 06:25:08.692458 10367 raft_consensus.cc:2781] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:08.692615 10367 raft_consensus.cc:482] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:08.692724 10367 raft_consensus.cc:3037] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:08.695176 10367 raft_consensus.cc:504] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } }
I20260809 06:25:08.695878 10367 leader_election.cc:288] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [CANDIDATE]: Term 1 election: Requested vote from peers 468600403a7343d1b03daa38ca9b0166 (127.8.131.3:44101), ddf626f484ab475d8c43fd6274b7fc83 (127.8.131.2:43373)
I20260809 06:25:08.696331 10265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b45ee26dc47b41499826845380018b24" candidate_uuid: "0b2af47867c748e18a8d1480c7c904a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "468600403a7343d1b03daa38ca9b0166"
I20260809 06:25:08.696547 10265 raft_consensus.cc:3037] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:08.698771 10265 raft_consensus.cc:2454] T b45ee26dc47b41499826845380018b24 P 468600403a7343d1b03daa38ca9b0166 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2af47867c748e18a8d1480c7c904a9 in term 1.
I20260809 06:25:08.699110 10064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b45ee26dc47b41499826845380018b24" candidate_uuid: "0b2af47867c748e18a8d1480c7c904a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf626f484ab475d8c43fd6274b7fc83"
I20260809 06:25:08.699193 9752 leader_election.cc:302] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2af47867c748e18a8d1480c7c904a9, 468600403a7343d1b03daa38ca9b0166; no voters:
I20260809 06:25:08.699324 10064 raft_consensus.cc:3037] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:08.699507 10367 raft_consensus.cc:2781] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:08.700037 10367 raft_consensus.cc:686] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [term 1 LEADER]: Becoming Leader. State: Replica: 0b2af47867c748e18a8d1480c7c904a9, State: Running, Role: LEADER
I20260809 06:25:08.700569 10367 consensus_queue.cc:237] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } }
I20260809 06:25:08.702543 10064 raft_consensus.cc:2454] T b45ee26dc47b41499826845380018b24 P ddf626f484ab475d8c43fd6274b7fc83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2af47867c748e18a8d1480c7c904a9 in term 1.
I20260809 06:25:08.705715 9626 catalog_manager.cc:5495] T b45ee26dc47b41499826845380018b24 P 0b2af47867c748e18a8d1480c7c904a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b2af47867c748e18a8d1480c7c904a9 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "0b2af47867c748e18a8d1480c7c904a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2af47867c748e18a8d1480c7c904a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "468600403a7343d1b03daa38ca9b0166" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddf626f484ab475d8c43fd6274b7fc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43373 } health_report { overall_health: UNKNOWN } } }
Aug 09 06:25:08 dist-test-slave-mjjr krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:08.774362 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.759345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39938 (local address 127.8.131.1:41583)
0809 06:25:08.759550 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.759555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.760169 (+ 614us) server_negotiation.cc:407] Connection header received
0809 06:25:08.760423 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.760427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.760504 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.760611 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:08.762136 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.762875 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.764451 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.764870 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.765011 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.769157 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:08.769172 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:08.769187 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:08.769224 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:08.772026 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.772644 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.772651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.772653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.772792 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.773129 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.773132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.773134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.773475 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:08.773717 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.773953 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.774124 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 06:25:08 dist-test-slave-mjjr krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 06:25:08 dist-test-slave-mjjr krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:08.824436 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.809375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39952 (local address 127.8.131.1:41583)
0809 06:25:08.809621 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.809628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.809973 (+ 345us) server_negotiation.cc:407] Connection header received
0809 06:25:08.810192 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.810196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.810262 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.810379 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:08.811971 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.812901 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.814292 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.814712 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.814853 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.819158 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:08.819163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:08.819166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:08.819200 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:08.821791 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.822436 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.822444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.822448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.822582 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.822966 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.822971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.822975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.823235 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:08.823548 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.823875 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.824100 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 06:25:08 dist-test-slave-mjjr krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 06:25:08 dist-test-slave-mjjr krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:08.873039 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.859031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39954 (local address 127.8.131.1:41583)
0809 06:25:08.859271 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.859279 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.859583 (+ 304us) server_negotiation.cc:407] Connection header received
0809 06:25:08.859868 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.859872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.859952 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.860064 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:08.861421 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.862179 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.863433 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.863931 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.864078 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.868119 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:08.868124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:08.868128 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:08.868160 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:08.870762 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.871318 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.871322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.871324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.871449 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.871817 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.871820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.871822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.872084 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:08.872277 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.872489 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.872643 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 06:25:08.874821 9786 tablet_service.cc:2908] Rejecting scan request for tablet b45ee26dc47b41499826845380018b24: Uninitialized: clean time has not yet been initialized
I20260809 06:25:08.877791 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9723
I20260809 06:25:08.889559 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:08.878677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.8.131.1:41583)
0809 06:25:08.878832 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:08.878836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:08.879244 (+ 408us) server_negotiation.cc:407] Connection header received
0809 06:25:08.879363 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:08.879366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:08.879418 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:08.879494 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:08.880682 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.881391 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.882503 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:08.882877 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:08.882963 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:08.885102 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:08.885106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:08.885109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:08.885135 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:08.887394 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.888032 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.888036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.888038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.888153 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:08.888462 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:08.888465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:08.888467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:08.888688 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:08.888859 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:08.889111 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 06:25:08.889285 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 06:25:08.890161 9910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:08.983979 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9723
I20260809 06:25:09.010648 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9931
Aug 09 06:25:09 dist-test-slave-mjjr krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:25:09.025609 10358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:09.011481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.8.131.2:43373)
0809 06:25:09.011683 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:09.011691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:09.011714 (+ 23us) server_negotiation.cc:407] Connection header received
0809 06:25:09.011963 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:09.011968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:09.012041 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:09.012157 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:09.013370 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:09.014097 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:09.015388 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:09.015905 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:09.016051 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:09.019929 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:09.019943 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:09.019959 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:09.019991 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:09.022581 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:09.023241 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:09.023256 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:09.023259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:09.023417 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:09.023908 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:09.023911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:09.023913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:09.024259 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:09.024587 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:09.024919 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 06:25:09.025170 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 06:25:09.026436 10118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:09.113936 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9931
I20260809 06:25:09.140141 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 10133
Aug 09 06:25:09 dist-test-slave-mjjr krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:25:09.155784 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:09.141153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57686 (local address 127.8.131.3:44101)
0809 06:25:09.141364 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:09.141370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:09.141392 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:25:09.141586 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:09.141590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:09.141656 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:09.141765 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:09.143041 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:09.143823 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:09.145161 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:09.145589 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:09.145731 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:09.150163 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:09.150179 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:09.150194 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:09.150235 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:09.152839 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:09.153551 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:09.153559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:09.153561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:09.153711 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:09.154093 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:09.154099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:09.154102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:09.154462 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:09.154770 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:09.155118 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 06:25:09.155351 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 06:25:09.156432 10321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:09.245018 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 10133
I20260809 06:25:09.267935 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9574
I20260809 06:25:09.269117 9670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:09.377574 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9574
2026-08-09T06:25:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4733 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:09 dist-test-slave-mjjr krb5kdc[10404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:09 dist-test-slave-mjjr krb5kdc[10404](info): set up 2 sockets
Aug 09 06:25:09 dist-test-slave-mjjr krb5kdc[10404](info): commencing operation
krb5kdc: starting...
W20260809 06:25:10.665000 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:10 dist-test-slave-mjjr krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:10.857226 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41235
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:41235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:11.182843 10426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:11.183120 10426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:11.183275 10426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:11.190514 10426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:11.190613 10426 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 06:25:11.190688 10426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:11.190802 10426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:11.190882 10426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:11.190989 10426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:11.200749 10426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:41235
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:11.202015 10426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:11.203363 10426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:11.215080 10435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:11.215342 10432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:11.215333 10433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:11.216444 10426 server_base.cc:1034] running on GCE node
I20260809 06:25:11.217275 10426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:11.218604 10426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:11.219846 10426 hybrid_clock.cc:648] HybridClock initialized: now 1786256711219778 us; error 58 us; skew 500 ppm
I20260809 06:25:11.220168 10426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:11 dist-test-slave-mjjr krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256711, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:11.223188 10426 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:11.224601 10426 webserver.cc:460] Webserver started at http://127.8.131.62:40009/ using document root <none> and password file <none>
I20260809 06:25:11.225291 10426 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:11.225423 10426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:11.225752 10426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:11.228746 10426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "0960f738e3ef41888ac3fbc3adf0a02d"
format_stamp: "Formatted at 2026-08-09 06:25:11 on dist-test-slave-mjjr"
server_key: "8931fefc450f41f8c5b71885445141b6"
server_key_iv: "6c58bf05afe43a03830d121a367dcf0b"
server_key_version: "encryptionkey@0"
I20260809 06:25:11.229552 10426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "0960f738e3ef41888ac3fbc3adf0a02d"
format_stamp: "Formatted at 2026-08-09 06:25:11 on dist-test-slave-mjjr"
server_key: "8931fefc450f41f8c5b71885445141b6"
server_key_iv: "6c58bf05afe43a03830d121a367dcf0b"
server_key_version: "encryptionkey@0"
I20260809 06:25:11.234532 10426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:11.237926 10445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:11.239176 10426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:25:11.239360 10426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "0960f738e3ef41888ac3fbc3adf0a02d"
format_stamp: "Formatted at 2026-08-09 06:25:11 on dist-test-slave-mjjr"
server_key: "8931fefc450f41f8c5b71885445141b6"
server_key_iv: "6c58bf05afe43a03830d121a367dcf0b"
server_key_version: "encryptionkey@0"
I20260809 06:25:11.239589 10426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:11.256443 10426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:11.260529 10426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:11.260810 10426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:11.282320 10426 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:41235
I20260809 06:25:11.282346 10544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:41235 every 8 connection(s)
I20260809 06:25:11.283715 10426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:11.286640 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 10426
I20260809 06:25:11.286890 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:11.287480 10545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:11.287541 8716 external_mini_cluster.cc:1428] Setting key a31bd4d66f256bd2ef9d32af6e7b6b9c
I20260809 06:25:11.301964 10545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d: Bootstrap starting.
Aug 09 06:25:11 dist-test-slave-mjjr krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:11.305779 10545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:11.306917 10545 log.cc:824] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:11.310303 10545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d: No bootstrap required, opened a new log
I20260809 06:25:11.311015 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:11.289550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.8.131.62:41235)
0809 06:25:11.290397 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:11.290417 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:11.290469 (+ 52us) server_negotiation.cc:407] Connection header received
0809 06:25:11.292464 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:11.292493 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:11.292985 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:11.293408 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:11.294666 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.296706 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.297992 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.299067 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.299264 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:11.303631 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:11.303680 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:11.303703 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:11.303749 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:11.306761 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:11.307606 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:11.307617 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:11.307627 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:11.307891 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:11.308260 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:11.308265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:11.308268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:11.308848 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:11.309066 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:11.309552 (+ 486us) server_negotiation.cc:299] Negotiation successful
0809 06:25:11.309924 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":365,"threads_started":1}
I20260809 06:25:11.319123 10545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0960f738e3ef41888ac3fbc3adf0a02d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41235 } }
I20260809 06:25:11.319509 10545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:11.319628 10545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0960f738e3ef41888ac3fbc3adf0a02d, State: Initialized, Role: FOLLOWER
I20260809 06:25:11.320264 10545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0960f738e3ef41888ac3fbc3adf0a02d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41235 } }
I20260809 06:25:11.320514 10545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:11.320647 10545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:11.320868 10545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:11.323257 10545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0960f738e3ef41888ac3fbc3adf0a02d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41235 } }
I20260809 06:25:11.323868 10545 leader_election.cc:302] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0960f738e3ef41888ac3fbc3adf0a02d; no voters:
I20260809 06:25:11.324440 10545 leader_election.cc:288] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:11.324575 10557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:11.325310 10557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [term 1 LEADER]: Becoming Leader. State: Replica: 0960f738e3ef41888ac3fbc3adf0a02d, State: Running, Role: LEADER
I20260809 06:25:11.325807 10557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0960f738e3ef41888ac3fbc3adf0a02d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41235 } }
I20260809 06:25:11.326524 10545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:11.328648 10558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0960f738e3ef41888ac3fbc3adf0a02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0960f738e3ef41888ac3fbc3adf0a02d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41235 } } }
I20260809 06:25:11.328858 10559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0960f738e3ef41888ac3fbc3adf0a02d. Latest consensus state: current_term: 1 leader_uuid: "0960f738e3ef41888ac3fbc3adf0a02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0960f738e3ef41888ac3fbc3adf0a02d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41235 } } }
I20260809 06:25:11.329098 10559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:11.329077 10558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:11.333303 10564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:11.338719 10564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:11.347795 10564 catalog_manager.cc:1347] Generated new cluster ID: f55cbbfc0cd347eabedd3257dec7f8de
I20260809 06:25:11.347919 10564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:11.362154 10564 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:11.363401 10564 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:11.377837 10564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d: Generated new TSK 0
I20260809 06:25:11.378860 10564 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:25:11.428666 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:41235
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:11.755548 10586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:11.755885 10586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:11.756067 10586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:11.763130 10586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:11.763232 10586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:11.763312 10586 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 06:25:11.763476 10586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:11.773382 10586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.131.62:41235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:11.774703 10586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:11.776049 10586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:11.789124 10594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:11.789161 10598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:11.789206 10595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:11.790309 10586 server_base.cc:1034] running on GCE node
I20260809 06:25:11.790920 10586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:11.791843 10586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:11.793111 10586 hybrid_clock.cc:648] HybridClock initialized: now 1786256711793037 us; error 40 us; skew 500 ppm
I20260809 06:25:11.793439 10586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:11 dist-test-slave-mjjr krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256711, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:11.802078 10586 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:25:11.803510 10586 webserver.cc:460] Webserver started at http://127.8.131.1:38907/ using document root <none> and password file <none>
I20260809 06:25:11.804315 10586 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:11.804445 10586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:11.804771 10586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:11.807576 10586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "15685ffdd66f4a19885ba3ad33719ed7"
format_stamp: "Formatted at 2026-08-09 06:25:11 on dist-test-slave-mjjr"
server_key: "7890c829832824c4fde56f7e43be5432"
server_key_iv: "b77f1d21abca1bb1082c511211168e4f"
server_key_version: "encryptionkey@0"
I20260809 06:25:11.808429 10586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "15685ffdd66f4a19885ba3ad33719ed7"
format_stamp: "Formatted at 2026-08-09 06:25:11 on dist-test-slave-mjjr"
server_key: "7890c829832824c4fde56f7e43be5432"
server_key_iv: "b77f1d21abca1bb1082c511211168e4f"
server_key_version: "encryptionkey@0"
I20260809 06:25:11.813376 10586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:11.816711 10615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:11.817932 10586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:11.818104 10586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "15685ffdd66f4a19885ba3ad33719ed7"
format_stamp: "Formatted at 2026-08-09 06:25:11 on dist-test-slave-mjjr"
server_key: "7890c829832824c4fde56f7e43be5432"
server_key_iv: "b77f1d21abca1bb1082c511211168e4f"
server_key_version: "encryptionkey@0"
I20260809 06:25:11.818316 10586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:11.840715 10586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:11.844487 10586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:11.844743 10586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:11.847390 10586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:11.847507 10586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:11.847687 10586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:11.847750 10586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 09 06:25:11 dist-test-slave-mjjr krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256711, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:11.870041 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:11.847765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:52877 (local address 127.8.131.62:41235)
0809 06:25:11.848229 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:11.848237 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:11.849422 (+ 1185us) server_negotiation.cc:407] Connection header received
0809 06:25:11.851529 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:11.851538 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:11.851673 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:11.851809 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:11.853769 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.854851 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.857478 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.858370 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.858519 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:11.863203 (+ 4684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:11.863210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:11.863214 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:11.863255 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:11.866240 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:11.866944 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:11.866950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:11.866953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:11.867084 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:11.867521 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:11.867526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:11.867534 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:11.867942 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:11.868088 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:11.869120 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0809 06:25:11.869448 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":277,"threads_started":1}
I20260809 06:25:11.871093 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:11.848472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41235 (local address 127.8.131.1:52877)
0809 06:25:11.849268 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:11.849311 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:11.851168 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:11.852022 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:11.852036 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:11.852736 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:11.853566 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:11.853601 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.855035 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.855039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:11.857230 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:11.857243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.858604 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.858613 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:11.858964 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:11.859715 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:11.859752 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:11.862968 (+ 3216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:11.866383 (+ 3415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:11.866391 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:11.866405 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:11.866815 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:11.867196 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:11.867201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:11.867205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:11.867396 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:11.868087 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:11.868129 (+ 42us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:11.868611 (+ 482us) client_negotiation.cc:769] Sending connection context
0809 06:25:11.868968 (+ 357us) client_negotiation.cc:240] Negotiation successful
0809 06:25:11.869471 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":342,"threads_started":1}
I20260809 06:25:11.896199 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:11.882029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:33591 (local address 127.8.131.62:41235)
0809 06:25:11.882176 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:11.882180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:11.882921 (+ 741us) server_negotiation.cc:407] Connection header received
0809 06:25:11.883107 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:11.883112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:11.883186 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:11.883282 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:11.884655 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.885714 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.887894 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.888503 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.888642 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:11.890422 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:11.890428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:11.890433 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:11.890473 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:11.893380 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:11.894221 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:11.894229 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:11.894235 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:11.894384 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:11.894738 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:11.894744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:11.894747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:11.895015 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:11.895163 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:11.895550 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 06:25:11.895819 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 06:25:11.896319 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:11.882127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41235 (local address 127.8.131.1:33591)
0809 06:25:11.882818 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:11.882834 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:11.882990 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:11.883307 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:11.883323 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:11.883848 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:11.884426 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:11.884444 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.885874 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.885878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:11.887740 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:11.887757 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:11.888731 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:11.888740 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:11.888914 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:11.889553 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:11.889571 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:11.890241 (+ 670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:11.893512 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:11.893517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:11.893521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:11.894097 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:11.894497 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:11.894502 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:11.894505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:11.894631 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:11.895129 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:11.895134 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:11.895314 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 06:25:11.895449 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 06:25:11.895819 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":407,"threads_started":1}
I20260809 06:25:11.908346 10586 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:34753
I20260809 06:25:11.908383 10809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:34753 every 8 connection(s)
I20260809 06:25:11.909555 10586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:11.911965 10810 heartbeater.cc:347] Connected to a master server at 127.8.131.62:41235
I20260809 06:25:11.912195 10810 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:11.912760 10810 heartbeater.cc:511] Master 127.8.131.62:41235 requested a full tablet report, sending...
I20260809 06:25:11.914573 10479 ts_manager.cc:194] Registered new tserver with Master: 15685ffdd66f4a19885ba3ad33719ed7 (127.8.131.1:34753)
I20260809 06:25:11.916126 10479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:52877
I20260809 06:25:11.917196 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 10586
I20260809 06:25:11.917433 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:11.918242 8716 external_mini_cluster.cc:1428] Setting key 52bae203a9020eeed7cf455469947e18
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:25:11.961056 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:41235
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:12.285005 10815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:12.285336 10815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:12.285526 10815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:12.292804 10815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:12.292950 10815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:12.293084 10815 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 06:25:12.293216 10815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:12.302882 10815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.131.62:41235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:12.304185 10815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:12.305524 10815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:12.318692 10825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:12.318795 10828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:12.318826 10826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:12.319809 10815 server_base.cc:1034] running on GCE node
I20260809 06:25:12.320494 10815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:12.321518 10815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:12.322786 10815 hybrid_clock.cc:648] HybridClock initialized: now 1786256712322688 us; error 67 us; skew 500 ppm
I20260809 06:25:12.323138 10815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:12 dist-test-slave-mjjr krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256712, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:12.326404 10815 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:25:12.327996 10815 webserver.cc:460] Webserver started at http://127.8.131.2:40825/ using document root <none> and password file <none>
I20260809 06:25:12.328707 10815 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:12.328845 10815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:12.329180 10815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:12.332201 10815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "c2ddc062f804456a94d49c91bc89ae76"
format_stamp: "Formatted at 2026-08-09 06:25:12 on dist-test-slave-mjjr"
server_key: "d10a2c88f86d2a710edd1675b473a0d5"
server_key_iv: "b38b3271f45e8f1f9bc5a1abd89c459d"
server_key_version: "encryptionkey@0"
I20260809 06:25:12.333029 10815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "c2ddc062f804456a94d49c91bc89ae76"
format_stamp: "Formatted at 2026-08-09 06:25:12 on dist-test-slave-mjjr"
server_key: "d10a2c88f86d2a710edd1675b473a0d5"
server_key_iv: "b38b3271f45e8f1f9bc5a1abd89c459d"
server_key_version: "encryptionkey@0"
I20260809 06:25:12.338399 10815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:25:12.341800 10840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:12.343171 10815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 06:25:12.343339 10815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "c2ddc062f804456a94d49c91bc89ae76"
format_stamp: "Formatted at 2026-08-09 06:25:12 on dist-test-slave-mjjr"
server_key: "d10a2c88f86d2a710edd1675b473a0d5"
server_key_iv: "b38b3271f45e8f1f9bc5a1abd89c459d"
server_key_version: "encryptionkey@0"
I20260809 06:25:12.343554 10815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:12.370666 10815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:12.374737 10815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:12.375034 10815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:12.377573 10815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:12.377686 10815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:12.377880 10815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:12.377967 10815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 06:25:12 dist-test-slave-mjjr krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256712, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:12.401372 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:12.378327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:59871 (local address 127.8.131.62:41235)
0809 06:25:12.378552 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:12.378558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:12.380279 (+ 1721us) server_negotiation.cc:407] Connection header received
0809 06:25:12.383500 (+ 3221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:12.383508 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:12.383608 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:12.383755 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:12.385933 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.386703 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.388818 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.389458 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.389595 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:12.394354 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:12.394362 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:12.394369 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:12.394419 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:12.398167 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:12.398963 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:12.398967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:12.398970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:12.399098 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:12.399554 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:12.399558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:12.399560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:12.399921 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:12.400083 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:12.400790 (+ 707us) server_negotiation.cc:299] Negotiation successful
0809 06:25:12.401023 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 06:25:12.402352 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:12.379166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41235 (local address 127.8.131.2:59871)
0809 06:25:12.380084 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:12.380141 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:12.383131 (+ 2990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:12.384005 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:12.384033 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:12.384804 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:12.385760 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:12.385782 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.386876 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.386880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:12.388593 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:12.388605 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.389715 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.389723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:12.389951 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:12.390727 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:12.390749 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:12.394072 (+ 3323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:12.398304 (+ 4232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:12.398312 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:12.398328 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:12.398754 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:12.399214 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:12.399219 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:12.399223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:12.399421 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:12.400062 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:12.400070 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:12.400428 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 06:25:12.400689 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 06:25:12.401193 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":400,"threads_started":1}
I20260809 06:25:12.424075 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:12.408038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41235 (local address 127.8.131.2:36581)
0809 06:25:12.408843 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:12.408881 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:12.409130 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:12.409547 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:12.409552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:12.410294 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:12.410784 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:12.410802 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.412282 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.412287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:12.413699 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:12.413713 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.414686 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.414696 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:12.414964 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:12.415818 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:12.415838 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:12.417106 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:12.420754 (+ 3648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:12.420761 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:12.420765 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:12.421419 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:12.421973 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:12.421980 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:12.421983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:12.422123 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:12.422970 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:12.422973 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:12.423177 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 06:25:12.423292 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 06:25:12.423520 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":485,"threads_started":1}
I20260809 06:25:12.424183 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:12.407926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:36581 (local address 127.8.131.62:41235)
0809 06:25:12.408119 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:12.408127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:12.409005 (+ 878us) server_negotiation.cc:407] Connection header received
0809 06:25:12.409322 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:12.409328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:12.409415 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:12.409521 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:12.411022 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.412154 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.413864 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.414433 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.414588 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:12.417380 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:12.417390 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:12.417394 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:12.417435 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:12.420624 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:12.421607 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:12.421615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:12.421621 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:12.421836 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:12.422235 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:12.422242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:12.422248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:12.422834 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:12.423023 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:12.423446 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 06:25:12.423772 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:25:12.434281 10815 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:40123
I20260809 06:25:12.434355 11023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:40123 every 8 connection(s)
I20260809 06:25:12.435516 10815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:12.437498 11025 heartbeater.cc:347] Connected to a master server at 127.8.131.62:41235
I20260809 06:25:12.437718 11025 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:12.438318 11025 heartbeater.cc:511] Master 127.8.131.62:41235 requested a full tablet report, sending...
I20260809 06:25:12.439617 10479 ts_manager.cc:194] Registered new tserver with Master: c2ddc062f804456a94d49c91bc89ae76 (127.8.131.2:40123)
I20260809 06:25:12.440011 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 10815
I20260809 06:25:12.440231 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:12.440562 10479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:59871
I20260809 06:25:12.440737 8716 external_mini_cluster.cc:1428] Setting key fb2006a2d247005b24f73c5f9e598aff
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:25:12.482370 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:41235
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:12.852099 11029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:12.852402 11029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:12.852581 11029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:12.859694 11029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:12.859814 11029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:12.859953 11029 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 06:25:12.860085 11029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:12.869592 11029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.131.62:41235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:12.870820 11029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:12.872166 11029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:12.885366 11039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:12.885457 11035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:12.885432 11036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:12.886566 11029 server_base.cc:1034] running on GCE node
I20260809 06:25:12.887163 11029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:12.888091 11029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:12.889314 11029 hybrid_clock.cc:648] HybridClock initialized: now 1786256712889252 us; error 31 us; skew 500 ppm
I20260809 06:25:12.889607 11029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:12 dist-test-slave-mjjr krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256712, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:12.892643 11029 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:25:12.894105 11029 webserver.cc:460] Webserver started at http://127.8.131.3:35697/ using document root <none> and password file <none>
I20260809 06:25:12.894784 11029 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:12.894925 11029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:12.895251 11029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:12.898209 11029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "9ab31f6549c144c88366ffe8d27c68ff"
format_stamp: "Formatted at 2026-08-09 06:25:12 on dist-test-slave-mjjr"
server_key: "1951c389dc7b9f670fcae3d04167696a"
server_key_iv: "cdd3a2df9848bc7971f022aec4825a5b"
server_key_version: "encryptionkey@0"
I20260809 06:25:12.899057 11029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "9ab31f6549c144c88366ffe8d27c68ff"
format_stamp: "Formatted at 2026-08-09 06:25:12 on dist-test-slave-mjjr"
server_key: "1951c389dc7b9f670fcae3d04167696a"
server_key_iv: "cdd3a2df9848bc7971f022aec4825a5b"
server_key_version: "encryptionkey@0"
I20260809 06:25:12.904363 11029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 06:25:12.908110 11054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:12.909525 11029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:25:12.909693 11029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "9ab31f6549c144c88366ffe8d27c68ff"
format_stamp: "Formatted at 2026-08-09 06:25:12 on dist-test-slave-mjjr"
server_key: "1951c389dc7b9f670fcae3d04167696a"
server_key_iv: "cdd3a2df9848bc7971f022aec4825a5b"
server_key_version: "encryptionkey@0"
I20260809 06:25:12.909929 11029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:12.919270 10810 heartbeater.cc:503] Master 127.8.131.62:41235 was elected leader, sending a full tablet report...
I20260809 06:25:12.923722 11029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:12.928145 11029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:12.928380 11029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:12.931042 11029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:12.931161 11029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:12.931316 11029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:12.931401 11029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 06:25:12 dist-test-slave-mjjr krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256712, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:12.958999 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:12.931811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:36225 (local address 127.8.131.62:41235)
0809 06:25:12.934455 (+ 2644us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:12.934462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:12.934486 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:25:12.937309 (+ 2823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:12.937315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:12.937411 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:12.937537 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:12.940222 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.941262 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.944658 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.945083 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.945186 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:12.950671 (+ 5485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:12.950676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:12.950679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:12.950713 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:12.954974 (+ 4261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:12.956246 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:12.956250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:12.956252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:12.956387 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:12.957007 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:12.957011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:12.957013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:12.957292 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:12.957425 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:12.958486 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0809 06:25:12.958671 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2509,"thread_start_us":2398,"threads_started":1}
I20260809 06:25:12.961201 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:12.932628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41235 (local address 127.8.131.3:36225)
0809 06:25:12.933883 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:12.933955 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:12.936836 (+ 2881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:12.937915 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:12.937932 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:12.938911 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:12.939993 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:12.940032 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.941458 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.941465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:12.944401 (+ 2936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:12.944442 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.945355 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.945366 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:12.945725 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:12.946718 (+ 993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:12.946760 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:12.950327 (+ 3567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:12.955181 (+ 4854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:12.955196 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:12.955217 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:12.956077 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:12.956508 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:12.956515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:12.956521 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:12.956879 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:12.957433 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:12.957446 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:12.958013 (+ 567us) client_negotiation.cc:769] Sending connection context
0809 06:25:12.958387 (+ 374us) client_negotiation.cc:240] Negotiation successful
0809 06:25:12.959113 (+ 726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":427,"threads_started":1}
I20260809 06:25:12.979741 11029 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:41779
I20260809 06:25:12.979741 11240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:41779 every 8 connection(s)
I20260809 06:25:12.981083 11029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:12.982259 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 11029
I20260809 06:25:12.982487 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:12.983108 8716 external_mini_cluster.cc:1428] Setting key 337be9a3f651b54d25e0c9fa6b4d4340
I20260809 06:25:12.984359 11241 heartbeater.cc:347] Connected to a master server at 127.8.131.62:41235
I20260809 06:25:12.984854 11241 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:12.985790 11241 heartbeater.cc:511] Master 127.8.131.62:41235 requested a full tablet report, sending...
I20260809 06:25:12.987566 10479 ts_manager.cc:194] Registered new tserver with Master: 9ab31f6549c144c88366ffe8d27c68ff (127.8.131.3:41779)
I20260809 06:25:12.988646 10479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:36225
I20260809 06:25:12.989496 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:12.974097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:49501 (local address 127.8.131.62:41235)
0809 06:25:12.974331 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:12.974338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:12.974960 (+ 622us) server_negotiation.cc:407] Connection header received
0809 06:25:12.975134 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:12.975143 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:12.975268 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:12.975402 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:12.976644 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.977682 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.979163 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.979609 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.979781 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:12.981736 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:12.981746 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:12.981751 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:12.981784 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:12.985703 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:12.986713 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:12.986718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:12.986722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:12.986901 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:12.987739 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:12.987744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:12.987747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:12.988211 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:12.988367 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:12.988893 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 06:25:12.989152 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":60}
I20260809 06:25:12.989460 11221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:12.974100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41235 (local address 127.8.131.3:49501)
0809 06:25:12.974852 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:12.974892 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:12.975011 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:12.975423 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:12.975426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:12.975960 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:12.976479 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:12.976497 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.977818 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.977824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:12.979017 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:12.979029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:12.979843 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:12.979851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:12.980042 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:12.980681 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:12.980698 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:12.981537 (+ 839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:12.985916 (+ 4379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:12.985921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:12.985924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:12.986503 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:12.987327 (+ 824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:12.987333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:12.987340 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:12.987547 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:12.988346 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:12.988354 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:12.988583 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 06:25:12.988749 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 06:25:12.989114 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":417,"threads_started":1}
I20260809 06:25:12.998678 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 06:25:13 dist-test-slave-mjjr krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 06:25:13 dist-test-slave-mjjr krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:13.048054 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.034238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42894 (local address 127.8.131.62:41235)
0809 06:25:13.034420 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.034453 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.034821 (+ 368us) server_negotiation.cc:407] Connection header received
0809 06:25:13.035040 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.035045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.035127 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.035214 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:13.036637 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.037358 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.038569 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.038981 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.039070 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.042737 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:13.042742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:13.042745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:13.042777 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:13.045593 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:13.046136 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:13.046140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:13.046142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:13.046265 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:13.046610 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:13.046613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:13.046615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:13.046946 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:13.047161 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.047431 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.047588 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:25:13.052080 10479 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:25:13.058851 10479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:13.084952 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.070916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:41779 (local address 127.0.0.1:53148)
0809 06:25:13.072830 (+ 1914us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:13.072867 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:13.073038 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:13.076778 (+ 3740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:13.076785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:13.076821 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:13.077258 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.077274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.080364 (+ 3090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.080371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.082082 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.082098 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.083829 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.083836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.084044 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.084104 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 06:25:13.084285 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 06:25:13.084489 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1678,"mutex_wait_us":42,"thread_start_us":1516,"threads_started":1}
I20260809 06:25:13.085053 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.072450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36862 (local address 127.8.131.1:34753)
0809 06:25:13.074196 (+ 1746us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.074205 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.074234 (+ 29us) server_negotiation.cc:407] Connection header received
0809 06:25:13.077030 (+ 2796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.077039 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.077258 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.077523 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:13.078255 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.079524 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.082584 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.084058 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.084197 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.084383 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.084651 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.084786 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1582,"thread_start_us":1417,"threads_started":1}
I20260809 06:25:13.085124 11266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.071206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:34753 (local address 127.0.0.1:36862)
0809 06:25:13.073542 (+ 2336us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:13.073555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:13.073675 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:13.077518 (+ 3843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:13.077522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:13.077548 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:13.078039 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.078052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.079676 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.079684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.082369 (+ 2685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.082385 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.084268 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.084274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.084482 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.084509 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 06:25:13.084603 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 06:25:13.084747 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2233,"mutex_wait_us":21,"thread_start_us":2132,"threads_started":1}
I20260809 06:25:13.085733 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.071898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53148 (local address 127.8.131.3:41779)
0809 06:25:13.076210 (+ 4312us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.076219 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.076251 (+ 32us) server_negotiation.cc:407] Connection header received
0809 06:25:13.076352 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.076360 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.076568 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.076767 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:13.077487 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.080238 (+ 2751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.082244 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.083614 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.083802 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.084047 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.085221 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.085402 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4110,"thread_start_us":3862,"threads_started":1}
I20260809 06:25:13.087250 11274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.074044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:40123 (local address 127.0.0.1:32808)
0809 06:25:13.077027 (+ 2983us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:13.077044 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:13.077177 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:13.077860 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:13.077867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:13.077897 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:13.078291 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.078302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.080144 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.080149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.082421 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.082434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.085625 (+ 3191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.085631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.085805 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.086650 (+ 845us) client_negotiation.cc:769] Sending connection context
0809 06:25:13.086747 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 06:25:13.086878 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2833,"mutex_wait_us":35,"thread_start_us":2691,"threads_started":1}
I20260809 06:25:13.087328 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.074170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.8.131.2:40123)
0809 06:25:13.076525 (+ 2355us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.076537 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.077153 (+ 616us) server_negotiation.cc:407] Connection header received
0809 06:25:13.077360 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.077369 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.077635 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.077840 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:13.078506 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.079994 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.082592 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.084058 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.084199 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.084380 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.086849 (+ 2469us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.087031 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2142,"thread_start_us":1980,"threads_started":1}
I20260809 06:25:13.089212 10705 tablet_service.cc:1467] Processing CreateTablet for tablet c1b42a5bd95c4cca852ee02485521c23 (DEFAULT_TABLE table=test-table [id=6bb088ff9c1543279060820d01ad9c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:13.089304 11129 tablet_service.cc:1467] Processing CreateTablet for tablet c1b42a5bd95c4cca852ee02485521c23 (DEFAULT_TABLE table=test-table [id=6bb088ff9c1543279060820d01ad9c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:13.090459 10705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1b42a5bd95c4cca852ee02485521c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:13.090742 11129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1b42a5bd95c4cca852ee02485521c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:13.091038 10924 tablet_service.cc:1467] Processing CreateTablet for tablet c1b42a5bd95c4cca852ee02485521c23 (DEFAULT_TABLE table=test-table [id=6bb088ff9c1543279060820d01ad9c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:13.092370 10924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1b42a5bd95c4cca852ee02485521c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:13.102855 11276 tablet_bootstrap.cc:492] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: Bootstrap starting.
I20260809 06:25:13.106176 11276 tablet_bootstrap.cc:654] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:13.107223 11276 log.cc:824] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:13.108201 11278 tablet_bootstrap.cc:492] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: Bootstrap starting.
I20260809 06:25:13.109791 11279 tablet_bootstrap.cc:492] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: Bootstrap starting.
I20260809 06:25:13.112349 11278 tablet_bootstrap.cc:654] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:13.112972 11279 tablet_bootstrap.cc:654] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:13.113684 11278 log.cc:824] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:13.114058 11279 log.cc:824] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:13.114739 11276 tablet_bootstrap.cc:492] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: No bootstrap required, opened a new log
I20260809 06:25:13.115077 11276 ts_tablet_manager.cc:1397] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260809 06:25:13.122417 11278 tablet_bootstrap.cc:492] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: No bootstrap required, opened a new log
I20260809 06:25:13.122802 11278 ts_tablet_manager.cc:1397] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.014s
I20260809 06:25:13.123200 11279 tablet_bootstrap.cc:492] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: No bootstrap required, opened a new log
I20260809 06:25:13.123446 11279 ts_tablet_manager.cc:1397] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260809 06:25:13.127350 11276 raft_consensus.cc:348] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } }
I20260809 06:25:13.127820 11276 raft_consensus.cc:374] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:13.127929 11276 raft_consensus.cc:729] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15685ffdd66f4a19885ba3ad33719ed7, State: Initialized, Role: FOLLOWER
I20260809 06:25:13.128559 11276 consensus_queue.cc:260] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } }
I20260809 06:25:13.130384 11276 ts_tablet_manager.cc:1428] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260809 06:25:13.131975 11279 raft_consensus.cc:348] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } }
I20260809 06:25:13.132416 11279 raft_consensus.cc:374] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:13.132545 11279 raft_consensus.cc:729] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ddc062f804456a94d49c91bc89ae76, State: Initialized, Role: FOLLOWER
I20260809 06:25:13.133185 11278 raft_consensus.cc:348] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } }
I20260809 06:25:13.133198 11279 consensus_queue.cc:260] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } }
I20260809 06:25:13.133545 11278 raft_consensus.cc:374] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:13.133646 11278 raft_consensus.cc:729] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab31f6549c144c88366ffe8d27c68ff, State: Initialized, Role: FOLLOWER
I20260809 06:25:13.134370 11278 consensus_queue.cc:260] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } }
I20260809 06:25:13.135586 11241 heartbeater.cc:503] Master 127.8.131.62:41235 was elected leader, sending a full tablet report...
I20260809 06:25:13.136157 11278 ts_tablet_manager.cc:1428] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260809 06:25:13.137531 11025 heartbeater.cc:503] Master 127.8.131.62:41235 was elected leader, sending a full tablet report...
I20260809 06:25:13.138293 11279 ts_tablet_manager.cc:1428] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
W20260809 06:25:13.162216 10812 tablet.cc:2369] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:25:13.187458 11026 tablet.cc:2369] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:25:13.239290 11242 tablet.cc:2369] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:25:13.424119 11286 raft_consensus.cc:482] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:13.424499 11286 raft_consensus.cc:504] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } }
I20260809 06:25:13.426193 11286 leader_election.cc:288] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15685ffdd66f4a19885ba3ad33719ed7 (127.8.131.1:34753), c2ddc062f804456a94d49c91bc89ae76 (127.8.131.2:40123)
I20260809 06:25:13.432092 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.426667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:34753 (local address 127.8.131.3:37727)
0809 06:25:13.426868 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:13.426885 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:13.427004 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:13.427681 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:13.427688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:13.427723 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:13.427893 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.427901 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.428954 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.428960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.430229 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.430245 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.431375 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.431381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.431573 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.431597 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 06:25:13.431715 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 06:25:13.431874 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 06:25:13.432763 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.426616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:37727 (local address 127.8.131.1:34753)
0809 06:25:13.426873 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.426879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.426946 (+ 67us) server_negotiation.cc:407] Connection header received
0809 06:25:13.427212 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.427220 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.427313 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.427471 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:13.428045 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.428815 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.430392 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.431224 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.431364 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.431467 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.432339 (+ 872us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.432512 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 06:25:13.433336 11287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.427118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:40123 (local address 127.8.131.3:32911)
0809 06:25:13.427791 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:13.427806 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:13.427953 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:13.428279 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:13.428284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:13.428318 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:13.428671 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.428679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.429680 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.429685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.431137 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.431149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.432525 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.432531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.432710 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.432744 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:25:13.432856 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 06:25:13.432994 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":357,"threads_started":1}
I20260809 06:25:13.433394 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.427295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:32911 (local address 127.8.131.2:40123)
0809 06:25:13.427553 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.427559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.427903 (+ 344us) server_negotiation.cc:407] Connection header received
0809 06:25:13.428099 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.428104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.428163 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.428255 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:13.428841 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.429545 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.431282 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.432317 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.432444 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.432527 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.432933 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.433075 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260809 06:25:13.433867 10746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b42a5bd95c4cca852ee02485521c23" candidate_uuid: "9ab31f6549c144c88366ffe8d27c68ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15685ffdd66f4a19885ba3ad33719ed7" is_pre_election: true
I20260809 06:25:13.434466 10746 raft_consensus.cc:2454] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab31f6549c144c88366ffe8d27c68ff in term 0.
I20260809 06:25:13.434342 10955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b42a5bd95c4cca852ee02485521c23" candidate_uuid: "9ab31f6549c144c88366ffe8d27c68ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ddc062f804456a94d49c91bc89ae76" is_pre_election: true
I20260809 06:25:13.434926 10955 raft_consensus.cc:2454] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab31f6549c144c88366ffe8d27c68ff in term 0.
I20260809 06:25:13.435180 11055 leader_election.cc:302] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15685ffdd66f4a19885ba3ad33719ed7, 9ab31f6549c144c88366ffe8d27c68ff; no voters:
I20260809 06:25:13.435566 11286 raft_consensus.cc:2781] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:13.435741 11286 raft_consensus.cc:482] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:13.435866 11286 raft_consensus.cc:3037] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:13.438436 11286 raft_consensus.cc:504] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } }
I20260809 06:25:13.439091 11286 leader_election.cc:288] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [CANDIDATE]: Term 1 election: Requested vote from peers 15685ffdd66f4a19885ba3ad33719ed7 (127.8.131.1:34753), c2ddc062f804456a94d49c91bc89ae76 (127.8.131.2:40123)
I20260809 06:25:13.439513 10746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b42a5bd95c4cca852ee02485521c23" candidate_uuid: "9ab31f6549c144c88366ffe8d27c68ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15685ffdd66f4a19885ba3ad33719ed7"
I20260809 06:25:13.439729 10746 raft_consensus.cc:3037] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:13.441901 10746 raft_consensus.cc:2454] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab31f6549c144c88366ffe8d27c68ff in term 1.
I20260809 06:25:13.442301 10955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b42a5bd95c4cca852ee02485521c23" candidate_uuid: "9ab31f6549c144c88366ffe8d27c68ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ddc062f804456a94d49c91bc89ae76"
I20260809 06:25:13.442328 11055 leader_election.cc:302] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15685ffdd66f4a19885ba3ad33719ed7, 9ab31f6549c144c88366ffe8d27c68ff; no voters:
I20260809 06:25:13.442526 10955 raft_consensus.cc:3037] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:13.442636 11286 raft_consensus.cc:2781] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:13.443178 11286 raft_consensus.cc:686] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 LEADER]: Becoming Leader. State: Replica: 9ab31f6549c144c88366ffe8d27c68ff, State: Running, Role: LEADER
I20260809 06:25:13.444190 11286 consensus_queue.cc:237] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } }
I20260809 06:25:13.446018 10955 raft_consensus.cc:2454] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab31f6549c144c88366ffe8d27c68ff in term 1.
I20260809 06:25:13.449364 10477 catalog_manager.cc:5495] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ab31f6549c144c88366ffe8d27c68ff (127.8.131.3). New cstate: current_term: 1 leader_uuid: "9ab31f6549c144c88366ffe8d27c68ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } health_report { overall_health: UNKNOWN } } }
I20260809 06:25:13.544948 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.539337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53164 (local address 127.8.131.3:41779)
0809 06:25:13.539545 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.539552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.539571 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:25:13.539943 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.539947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.540033 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.540171 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:13.540568 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.541257 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.542621 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.543071 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.543201 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.543673 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:13.544271 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:13.544385 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.544576 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.544651 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:25:13.552568 10955 raft_consensus.cc:1261] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Refusing update from remote peer 9ab31f6549c144c88366ffe8d27c68ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:13.552629 10746 raft_consensus.cc:1261] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 FOLLOWER]: Refusing update from remote peer 9ab31f6549c144c88366ffe8d27c68ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:13.553486 11289 consensus_queue.cc:1035] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:13.553838 11286 consensus_queue.cc:1035] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:13.586061 10479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:13.586319 10479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:13.589779 10479 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d: Sending DeleteTablet for 3 replicas of tablet c1b42a5bd95c4cca852ee02485521c23
I20260809 06:25:13.590962 11129 tablet_service.cc:1514] Processing DeleteTablet for tablet c1b42a5bd95c4cca852ee02485521c23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:13 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:53148
I20260809 06:25:13.591044 10924 tablet_service.cc:1514] Processing DeleteTablet for tablet c1b42a5bd95c4cca852ee02485521c23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:13 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:32808
I20260809 06:25:13.591084 10705 tablet_service.cc:1514] Processing DeleteTablet for tablet c1b42a5bd95c4cca852ee02485521c23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:13 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:36862
I20260809 06:25:13.595041 10479 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:25:13.595916 10479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:13.605142 11126 tablet_service.cc:1467] Processing CreateTablet for tablet 7c070be3517043768198c581003c4e18 (DEFAULT_TABLE table=test-table [id=648a91034b664ad2ae897433cf4c2c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:13.605140 10704 tablet_service.cc:1467] Processing CreateTablet for tablet 7c070be3517043768198c581003c4e18 (DEFAULT_TABLE table=test-table [id=648a91034b664ad2ae897433cf4c2c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:13.605798 10704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c070be3517043768198c581003c4e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:13.605805 11126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c070be3517043768198c581003c4e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:13.607766 11316 tablet_replica.cc:330] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: stopping tablet replica
I20260809 06:25:13.608390 11316 raft_consensus.cc:2229] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:13.609091 11316 raft_consensus.cc:2258] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:13.610145 11317 tablet_replica.cc:330] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: stopping tablet replica
I20260809 06:25:13.612141 10923 tablet_service.cc:1467] Processing CreateTablet for tablet 7c070be3517043768198c581003c4e18 (DEFAULT_TABLE table=test-table [id=648a91034b664ad2ae897433cf4c2c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:13.612705 10923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c070be3517043768198c581003c4e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:13.612812 11276 tablet_bootstrap.cc:492] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: Bootstrap starting.
I20260809 06:25:13.614524 11276 tablet_bootstrap.cc:654] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:13.615259 11319 tablet_replica.cc:330] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: stopping tablet replica
I20260809 06:25:13.615818 11319 raft_consensus.cc:2229] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:13.616411 11319 raft_consensus.cc:2258] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:13.617673 11317 raft_consensus.cc:2229] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:25:13.618502 11317 raft_consensus.cc:2258] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:13.618681 11319 ts_tablet_manager.cc:1905] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:13.616416 11316 ts_tablet_manager.cc:1905] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:13.621380 11279 tablet_bootstrap.cc:492] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: Bootstrap starting.
I20260809 06:25:13.623169 11278 tablet_bootstrap.cc:492] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: Bootstrap starting.
I20260809 06:25:13.624855 11278 tablet_bootstrap.cc:654] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:13.626343 11279 tablet_bootstrap.cc:654] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:13.626541 11317 ts_tablet_manager.cc:1905] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:13.640558 11319 ts_tablet_manager.cc:1918] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:25:13.640769 11319 log.cc:1196] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/c1b42a5bd95c4cca852ee02485521c23
I20260809 06:25:13.641391 11319 ts_tablet_manager.cc:1939] T c1b42a5bd95c4cca852ee02485521c23 P 15685ffdd66f4a19885ba3ad33719ed7: Deleting consensus metadata
I20260809 06:25:13.642139 11279 tablet_bootstrap.cc:492] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: No bootstrap required, opened a new log
I20260809 06:25:13.642318 11279 ts_tablet_manager.cc:1397] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.008s
I20260809 06:25:13.643311 11279 raft_consensus.cc:348] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.643548 11279 raft_consensus.cc:374] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:13.643529 10447 catalog_manager.cc:4841] TS 15685ffdd66f4a19885ba3ad33719ed7 (127.8.131.1:34753): tablet c1b42a5bd95c4cca852ee02485521c23 (table test-table [id=6bb088ff9c1543279060820d01ad9c87]) successfully deleted
I20260809 06:25:13.643677 11279 raft_consensus.cc:729] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ddc062f804456a94d49c91bc89ae76, State: Initialized, Role: FOLLOWER
I20260809 06:25:13.644014 11279 consensus_queue.cc:260] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.644703 11276 tablet_bootstrap.cc:492] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: No bootstrap required, opened a new log
I20260809 06:25:13.644858 11276 ts_tablet_manager.cc:1397] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.000s
I20260809 06:25:13.645254 11316 ts_tablet_manager.cc:1918] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:25:13.645493 11316 log.cc:1196] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/c1b42a5bd95c4cca852ee02485521c23
I20260809 06:25:13.645676 11276 raft_consensus.cc:348] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.645908 11276 raft_consensus.cc:374] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:13.646009 11276 raft_consensus.cc:729] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15685ffdd66f4a19885ba3ad33719ed7, State: Initialized, Role: FOLLOWER
I20260809 06:25:13.646345 11316 ts_tablet_manager.cc:1939] T c1b42a5bd95c4cca852ee02485521c23 P c2ddc062f804456a94d49c91bc89ae76: Deleting consensus metadata
I20260809 06:25:13.648190 10447 catalog_manager.cc:4841] TS c2ddc062f804456a94d49c91bc89ae76 (127.8.131.2:40123): tablet c1b42a5bd95c4cca852ee02485521c23 (table test-table [id=6bb088ff9c1543279060820d01ad9c87]) successfully deleted
I20260809 06:25:13.648094 11276 consensus_queue.cc:260] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.648548 11279 ts_tablet_manager.cc:1428] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260809 06:25:13.648528 11317 ts_tablet_manager.cc:1918] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:25:13.648855 11317 log.cc:1196] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/c1b42a5bd95c4cca852ee02485521c23
I20260809 06:25:13.649121 11278 tablet_bootstrap.cc:492] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: No bootstrap required, opened a new log
I20260809 06:25:13.649299 11278 ts_tablet_manager.cc:1397] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.004s
I20260809 06:25:13.649690 11317 ts_tablet_manager.cc:1939] T c1b42a5bd95c4cca852ee02485521c23 P 9ab31f6549c144c88366ffe8d27c68ff: Deleting consensus metadata
I20260809 06:25:13.650123 11278 raft_consensus.cc:348] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.650348 11278 raft_consensus.cc:374] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:13.650454 11278 raft_consensus.cc:729] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab31f6549c144c88366ffe8d27c68ff, State: Initialized, Role: FOLLOWER
I20260809 06:25:13.650817 11278 consensus_queue.cc:260] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.651461 11278 ts_tablet_manager.cc:1428] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 06:25:13.651698 10450 catalog_manager.cc:4841] TS 9ab31f6549c144c88366ffe8d27c68ff (127.8.131.3:41779): tablet c1b42a5bd95c4cca852ee02485521c23 (table test-table [id=6bb088ff9c1543279060820d01ad9c87]) successfully deleted
I20260809 06:25:13.652052 11276 ts_tablet_manager.cc:1428] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260809 06:25:13.677603 11284 raft_consensus.cc:482] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:13.677829 11284 raft_consensus.cc:504] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.679481 11284 leader_election.cc:288] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2ddc062f804456a94d49c91bc89ae76 (127.8.131.2:40123), 9ab31f6549c144c88366ffe8d27c68ff (127.8.131.3:41779)
I20260809 06:25:13.688105 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.680330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:57693 (local address 127.8.131.2:40123)
0809 06:25:13.680500 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.680508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.680685 (+ 177us) server_negotiation.cc:407] Connection header received
0809 06:25:13.681405 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.681412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.681510 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.681574 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:13.682502 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.683676 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.686257 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.687072 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.687228 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.687334 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.687683 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.687822 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:25:13.688210 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.679798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:40123 (local address 127.8.131.1:57693)
0809 06:25:13.680575 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:13.680596 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:13.681199 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:13.681788 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:13.681793 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:13.681819 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:13.682299 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.682317 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.683808 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.683813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.686053 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.686068 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.687299 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.687306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.687474 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.687499 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 06:25:13.687603 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 06:25:13.687766 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":62,"thread_start_us":719,"threads_started":1}
I20260809 06:25:13.688839 10955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c070be3517043768198c581003c4e18" candidate_uuid: "15685ffdd66f4a19885ba3ad33719ed7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ddc062f804456a94d49c91bc89ae76" is_pre_election: true
I20260809 06:25:13.689114 10955 raft_consensus.cc:2454] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15685ffdd66f4a19885ba3ad33719ed7 in term 0.
I20260809 06:25:13.689399 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.680240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:53829 (local address 127.8.131.3:41779)
0809 06:25:13.680423 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.680430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.681391 (+ 961us) server_negotiation.cc:407] Connection header received
0809 06:25:13.681584 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.681594 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.681677 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.681774 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:13.682558 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.683343 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.686230 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.687568 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.687735 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.687836 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.688958 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.689094 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 06:25:13.689602 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.679911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:41779 (local address 127.8.131.1:53829)
0809 06:25:13.681287 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:13.681301 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:13.681448 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:13.681797 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:13.681801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:13.681849 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:13.682390 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.682401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.683481 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.683487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.686053 (+ 2566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.686067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.687990 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.687997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.688497 (+ 500us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.688535 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 06:25:13.688916 (+ 381us) client_negotiation.cc:240] Negotiation successful
0809 06:25:13.689139 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1261,"mutex_wait_us":128,"spinlock_wait_cycles":5376,"thread_start_us":1086,"threads_started":1}
I20260809 06:25:13.689981 10617 leader_election.cc:302] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15685ffdd66f4a19885ba3ad33719ed7, c2ddc062f804456a94d49c91bc89ae76; no voters:
I20260809 06:25:13.690435 11284 raft_consensus.cc:2781] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:13.690579 11284 raft_consensus.cc:482] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:13.690680 11284 raft_consensus.cc:3037] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:13.690593 11164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c070be3517043768198c581003c4e18" candidate_uuid: "15685ffdd66f4a19885ba3ad33719ed7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ab31f6549c144c88366ffe8d27c68ff" is_pre_election: true
I20260809 06:25:13.691082 11164 raft_consensus.cc:2454] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15685ffdd66f4a19885ba3ad33719ed7 in term 0.
I20260809 06:25:13.693882 11284 raft_consensus.cc:504] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.695102 11164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c070be3517043768198c581003c4e18" candidate_uuid: "15685ffdd66f4a19885ba3ad33719ed7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ab31f6549c144c88366ffe8d27c68ff"
I20260809 06:25:13.695137 10955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c070be3517043768198c581003c4e18" candidate_uuid: "15685ffdd66f4a19885ba3ad33719ed7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ddc062f804456a94d49c91bc89ae76"
I20260809 06:25:13.695325 11164 raft_consensus.cc:3037] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:13.695328 10955 raft_consensus.cc:3037] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:13.694669 11284 leader_election.cc:288] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [CANDIDATE]: Term 1 election: Requested vote from peers c2ddc062f804456a94d49c91bc89ae76 (127.8.131.2:40123), 9ab31f6549c144c88366ffe8d27c68ff (127.8.131.3:41779)
I20260809 06:25:13.698491 10955 raft_consensus.cc:2454] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15685ffdd66f4a19885ba3ad33719ed7 in term 1.
I20260809 06:25:13.698633 11164 raft_consensus.cc:2454] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15685ffdd66f4a19885ba3ad33719ed7 in term 1.
I20260809 06:25:13.698987 10617 leader_election.cc:302] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15685ffdd66f4a19885ba3ad33719ed7, c2ddc062f804456a94d49c91bc89ae76; no voters:
I20260809 06:25:13.699281 11284 raft_consensus.cc:2781] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:13.699844 11284 raft_consensus.cc:686] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 LEADER]: Becoming Leader. State: Replica: 15685ffdd66f4a19885ba3ad33719ed7, State: Running, Role: LEADER
I20260809 06:25:13.700234 11284 consensus_queue.cc:237] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.703528 10479 catalog_manager.cc:5495] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15685ffdd66f4a19885ba3ad33719ed7 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "15685ffdd66f4a19885ba3ad33719ed7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } health_report { overall_health: UNKNOWN } } }
Aug 09 06:25:13 dist-test-slave-mjjr krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256711, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:13.761894 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.746142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41235 (local address 127.0.0.1:42896)
0809 06:25:13.747078 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:13.747099 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:13.747267 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:13.747698 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:13.747704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:13.748603 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:13.748885 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.748897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.749947 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.749953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.751494 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.751514 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.752282 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.752288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.752489 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.753223 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:13.753257 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:13.755846 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:13.759487 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:13.759516 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:13.759522 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:13.759989 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:13.760379 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:13.760381 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:13.760384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:13.760535 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:13.761118 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:13.761124 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:13.761278 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 06:25:13.761406 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 06:25:13.761600 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":621,"threads_started":1}
I20260809 06:25:13.762234 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.746156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.8.131.62:41235)
0809 06:25:13.746896 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.746905 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.747188 (+ 283us) server_negotiation.cc:407] Connection header received
0809 06:25:13.747419 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.747427 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.747515 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.747608 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:13.749039 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.749800 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.751693 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.752107 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.752204 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.756055 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:13.756061 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:13.756067 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:13.756102 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:13.759349 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:13.760098 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:13.760103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:13.760107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:13.760275 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:13.760650 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:13.760655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:13.760659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:13.761000 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:13.761150 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.761556 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.761787 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":359,"threads_started":1}
I20260809 06:25:13.767719 10479 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:42896:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 06:25:13.768971 10479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:13.776154 10923 tablet_service.cc:1467] Processing CreateTablet for tablet 6c37f3b5cc334088a5376075ef954940 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a75f26935fd945cea9df9a8188eeaa46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 06:25:13.776261 10704 tablet_service.cc:1467] Processing CreateTablet for tablet 6c37f3b5cc334088a5376075ef954940 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a75f26935fd945cea9df9a8188eeaa46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 06:25:13.776729 10923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c37f3b5cc334088a5376075ef954940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:13.776839 10704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c37f3b5cc334088a5376075ef954940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:13.782712 11276 tablet_bootstrap.cc:492] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: Bootstrap starting.
I20260809 06:25:13.784209 11276 tablet_bootstrap.cc:654] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:13.784611 11126 tablet_service.cc:1467] Processing CreateTablet for tablet 6c37f3b5cc334088a5376075ef954940 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a75f26935fd945cea9df9a8188eeaa46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 06:25:13.785142 11126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c37f3b5cc334088a5376075ef954940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:13.788285 11279 tablet_bootstrap.cc:492] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: Bootstrap starting.
I20260809 06:25:13.790006 11279 tablet_bootstrap.cc:654] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:13.790047 11276 tablet_bootstrap.cc:492] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: No bootstrap required, opened a new log
I20260809 06:25:13.790199 11276 ts_tablet_manager.cc:1397] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260809 06:25:13.790979 11276 raft_consensus.cc:348] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.791235 11276 raft_consensus.cc:374] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:13.791320 11276 raft_consensus.cc:729] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15685ffdd66f4a19885ba3ad33719ed7, State: Initialized, Role: FOLLOWER
I20260809 06:25:13.791620 11276 consensus_queue.cc:260] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.792104 11284 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:13.792289 11276 ts_tablet_manager.cc:1428] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 06:25:13.792419 11284 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:13.794150 11278 tablet_bootstrap.cc:492] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: Bootstrap starting.
I20260809 06:25:13.796069 11278 tablet_bootstrap.cc:654] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:13.797366 11279 tablet_bootstrap.cc:492] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: No bootstrap required, opened a new log
I20260809 06:25:13.797541 11279 ts_tablet_manager.cc:1397] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260809 06:25:13.798300 11278 tablet_bootstrap.cc:492] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: No bootstrap required, opened a new log
I20260809 06:25:13.798312 11279 raft_consensus.cc:348] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.798491 11278 ts_tablet_manager.cc:1397] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:25:13.798556 11279 raft_consensus.cc:374] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:13.798635 11279 raft_consensus.cc:729] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ddc062f804456a94d49c91bc89ae76, State: Initialized, Role: FOLLOWER
I20260809 06:25:13.798888 11279 consensus_queue.cc:260] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.799252 11278 raft_consensus.cc:348] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.799484 11278 raft_consensus.cc:374] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:13.799595 11278 raft_consensus.cc:729] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab31f6549c144c88366ffe8d27c68ff, State: Initialized, Role: FOLLOWER
I20260809 06:25:13.799345 11285 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:13.799746 11285 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:13.799919 11278 consensus_queue.cc:260] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.800359 11306 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:13.800776 11306 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:13.801018 11279 ts_tablet_manager.cc:1428] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 06:25:13.802093 11278 ts_tablet_manager.cc:1428] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 06:25:13.914703 11306 raft_consensus.cc:482] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:13.915011 11306 raft_consensus.cc:504] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.915969 11306 leader_election.cc:288] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15685ffdd66f4a19885ba3ad33719ed7 (127.8.131.1:34753), c2ddc062f804456a94d49c91bc89ae76 (127.8.131.2:40123)
I20260809 06:25:13.916935 10746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c37f3b5cc334088a5376075ef954940" candidate_uuid: "9ab31f6549c144c88366ffe8d27c68ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15685ffdd66f4a19885ba3ad33719ed7" is_pre_election: true
I20260809 06:25:13.917001 10955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c37f3b5cc334088a5376075ef954940" candidate_uuid: "9ab31f6549c144c88366ffe8d27c68ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ddc062f804456a94d49c91bc89ae76" is_pre_election: true
I20260809 06:25:13.917191 10746 raft_consensus.cc:2454] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab31f6549c144c88366ffe8d27c68ff in term 0.
I20260809 06:25:13.917291 10955 raft_consensus.cc:2454] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ab31f6549c144c88366ffe8d27c68ff in term 0.
I20260809 06:25:13.917793 11055 leader_election.cc:302] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ab31f6549c144c88366ffe8d27c68ff, c2ddc062f804456a94d49c91bc89ae76; no voters:
I20260809 06:25:13.918108 11306 raft_consensus.cc:2781] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:13.918248 11306 raft_consensus.cc:482] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:13.918349 11306 raft_consensus.cc:3037] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:13.922019 11306 raft_consensus.cc:504] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.922839 11306 leader_election.cc:288] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [CANDIDATE]: Term 1 election: Requested vote from peers 15685ffdd66f4a19885ba3ad33719ed7 (127.8.131.1:34753), c2ddc062f804456a94d49c91bc89ae76 (127.8.131.2:40123)
I20260809 06:25:13.923345 10955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c37f3b5cc334088a5376075ef954940" candidate_uuid: "9ab31f6549c144c88366ffe8d27c68ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ddc062f804456a94d49c91bc89ae76"
I20260809 06:25:13.923380 10746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c37f3b5cc334088a5376075ef954940" candidate_uuid: "9ab31f6549c144c88366ffe8d27c68ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15685ffdd66f4a19885ba3ad33719ed7"
I20260809 06:25:13.923592 10746 raft_consensus.cc:3037] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:13.923609 10955 raft_consensus.cc:3037] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:13.926821 10746 raft_consensus.cc:2454] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab31f6549c144c88366ffe8d27c68ff in term 1.
I20260809 06:25:13.927029 10955 raft_consensus.cc:2454] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab31f6549c144c88366ffe8d27c68ff in term 1.
I20260809 06:25:13.927350 11055 leader_election.cc:302] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15685ffdd66f4a19885ba3ad33719ed7, 9ab31f6549c144c88366ffe8d27c68ff; no voters:
I20260809 06:25:13.927742 11286 raft_consensus.cc:2781] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:13.927901 11286 raft_consensus.cc:686] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 LEADER]: Becoming Leader. State: Replica: 9ab31f6549c144c88366ffe8d27c68ff, State: Running, Role: LEADER
I20260809 06:25:13.928162 11286 consensus_queue.cc:237] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } }
I20260809 06:25:13.932070 11306 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: TxnStatusTablet state changed. Reason: New leader 9ab31f6549c144c88366ffe8d27c68ff. Latest consensus state: current_term: 1 leader_uuid: "9ab31f6549c144c88366ffe8d27c68ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:13.932519 11306 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: This TxnStatusTablet replica's current role is: LEADER
I20260809 06:25:13.933210 11355 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 06:25:13.933456 11355 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 06:25:13.933377 10477 catalog_manager.cc:5495] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ab31f6549c144c88366ffe8d27c68ff (127.8.131.3). New cstate: current_term: 1 leader_uuid: "9ab31f6549c144c88366ffe8d27c68ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } health_report { overall_health: HEALTHY } } }
I20260809 06:25:13.983062 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.976654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53180 (local address 127.8.131.3:41779)
0809 06:25:13.976822 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:13.976827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:13.976874 (+ 47us) server_negotiation.cc:407] Connection header received
0809 06:25:13.977189 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:13.977192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:13.977256 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:13.977334 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:13.977823 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.978559 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.980645 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.981441 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.981575 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.982210 (+ 635us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:13.982363 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:13.982473 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:13.982681 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 06:25:13.982761 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 06:25:13.983448 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:13.976494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:41779 (local address 127.0.0.1:53180)
0809 06:25:13.976787 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:13.976814 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:13.976987 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:13.977413 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:13.977421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:13.977438 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:25:13.977668 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.977682 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.978706 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.978722 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.980463 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:13.980484 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:13.981639 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:13.981647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:13.981901 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:13.981985 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:25:13.982502 (+ 517us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:25:13.982532 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 06:25:13.982640 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 06:25:13.982818 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 06:25:13.987907 10746 raft_consensus.cc:1261] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 FOLLOWER]: Refusing update from remote peer 9ab31f6549c144c88366ffe8d27c68ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:13.988150 10955 raft_consensus.cc:1261] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Refusing update from remote peer 9ab31f6549c144c88366ffe8d27c68ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:13.988741 11306 consensus_queue.cc:1035] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:13.989141 11286 consensus_queue.cc:1035] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:14.001214 11284 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: TxnStatusTablet state changed. Reason: New leader 9ab31f6549c144c88366ffe8d27c68ff. Latest consensus state: current_term: 1 leader_uuid: "9ab31f6549c144c88366ffe8d27c68ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:14.001585 11284 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:14.004673 11285 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: TxnStatusTablet state changed. Reason: New leader 9ab31f6549c144c88366ffe8d27c68ff. Latest consensus state: current_term: 1 leader_uuid: "9ab31f6549c144c88366ffe8d27c68ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:14.005002 11285 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:14.005635 11295 mvcc.cc:205] Tried to move back new op lower bound from 7316507500487417856 to 7316507500252254208. Current Snapshot: MvccSnapshot[applied={T|T < 7316507500487417856}]
I20260809 06:25:14.009155 11306 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ab31f6549c144c88366ffe8d27c68ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:14.009479 11306 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: This TxnStatusTablet replica's current role is: LEADER
I20260809 06:25:14.009434 11296 mvcc.cc:205] Tried to move back new op lower bound from 7316507500487417856 to 7316507500252254208. Current Snapshot: MvccSnapshot[applied={T|T < 7316507500487417856}]
I20260809 06:25:14.010944 11284 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ab31f6549c144c88366ffe8d27c68ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:14.011276 11284 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P 15685ffdd66f4a19885ba3ad33719ed7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:14.011493 11285 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ab31f6549c144c88366ffe8d27c68ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:14.011752 11285 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P c2ddc062f804456a94d49c91bc89ae76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:14.012451 11286 tablet_replica.cc:439] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ab31f6549c144c88366ffe8d27c68ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15685ffdd66f4a19885ba3ad33719ed7" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 34753 } } peers { permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 } } peers { permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 } } }
I20260809 06:25:14.012773 11286 tablet_replica.cc:442] T 6c37f3b5cc334088a5376075ef954940 P 9ab31f6549c144c88366ffe8d27c68ff: This TxnStatusTablet replica's current role is: LEADER
I20260809 06:25:14.028478 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.020449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.8.131.1:34753)
0809 06:25:14.023053 (+ 2604us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:14.023057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:14.023077 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:25:14.023154 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:14.023158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:14.023235 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:14.023407 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:14.023857 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.024713 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.026025 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.026450 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.026601 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.027154 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:14.027791 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:14.028009 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:14.028062 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 06:25:14.028142 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2483,"thread_start_us":2353,"threads_started":1}
I20260809 06:25:14.047623 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.040765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:45739 (local address 127.8.131.3:41779)
0809 06:25:14.040990 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:14.040996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:14.041372 (+ 376us) server_negotiation.cc:407] Connection header received
0809 06:25:14.041598 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:14.041602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:14.041668 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:14.041752 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:14.042429 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.043407 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.045620 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.046066 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.046163 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.046762 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:14.046971 (+ 209us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:14.047048 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:14.047205 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 06:25:14.047287 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 06:25:14.047823 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.040561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:41779 (local address 127.8.131.1:45739)
0809 06:25:14.041253 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:14.041277 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:14.041459 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:14.041815 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:14.041820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:14.041835 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:25:14.042262 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:14.042281 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.043553 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.043559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:14.045453 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:14.045473 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.046262 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.046271 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:14.046490 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.046575 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:25:14.047073 (+ 498us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:25:14.047087 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 06:25:14.047183 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 06:25:14.047328 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":323,"threads_started":1}
I20260809 06:25:14.059419 10955 raft_consensus.cc:1261] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Refusing update from remote peer 15685ffdd66f4a19885ba3ad33719ed7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:14.059516 11164 raft_consensus.cc:1261] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 FOLLOWER]: Refusing update from remote peer 15685ffdd66f4a19885ba3ad33719ed7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:14.060319 11325 consensus_queue.cc:1035] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ab31f6549c144c88366ffe8d27c68ff" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:14.060693 11284 consensus_queue.cc:1035] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ddc062f804456a94d49c91bc89ae76" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:14.077517 11297 mvcc.cc:205] Tried to move back new op lower bound from 7316507500779868160 to 7316507499319799808. Current Snapshot: MvccSnapshot[applied={T|T < 7316507500779868160}]
I20260809 06:25:14.101125 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.093917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:43055 (local address 127.8.131.1:34753)
0809 06:25:14.094078 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:14.094083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:14.094919 (+ 836us) server_negotiation.cc:407] Connection header received
0809 06:25:14.095148 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:14.095154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:14.095239 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:14.095326 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:14.096053 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.096960 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.099084 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.099639 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.099800 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.100374 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:14.100482 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:14.100586 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:14.100704 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 06:25:14.100784 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 06:25:14.101260 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.093970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:34753 (local address 127.8.131.3:43055)
0809 06:25:14.094799 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:14.094821 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:14.094989 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:14.095347 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:14.095352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:14.095365 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:25:14.095882 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:14.095895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.097094 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.097100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:14.098635 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:14.098651 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.099866 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.099873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:14.100080 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.100158 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:25:14.100587 (+ 429us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:25:14.100600 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 06:25:14.100689 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 06:25:14.100835 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":431,"threads_started":1}
I20260809 06:25:14.142858 10477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:14.143172 10477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:14.147490 10477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0960f738e3ef41888ac3fbc3adf0a02d: Sending DeleteTablet for 3 replicas of tablet 7c070be3517043768198c581003c4e18
I20260809 06:25:14.148662 11126 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c070be3517043768198c581003c4e18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:14 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:53148
I20260809 06:25:14.149017 11317 tablet_replica.cc:330] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: stopping tablet replica
I20260809 06:25:14.149232 11317 raft_consensus.cc:2229] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:14.149471 11317 raft_consensus.cc:2258] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:14.150290 10923 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c070be3517043768198c581003c4e18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:14 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:32808
I20260809 06:25:14.150942 11383 tablet_replica.cc:330] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: stopping tablet replica
I20260809 06:25:14.151216 11383 raft_consensus.cc:2229] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:14.151398 11317 ts_tablet_manager.cc:1905] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:14.151547 11383 raft_consensus.cc:2258] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:14.152058 10704 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c070be3517043768198c581003c4e18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:14 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:36862
I20260809 06:25:14.152912 11383 ts_tablet_manager.cc:1905] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:14.160549 11383 ts_tablet_manager.cc:1918] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:25:14.160827 11383 log.cc:1196] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/7c070be3517043768198c581003c4e18
I20260809 06:25:14.161428 11383 ts_tablet_manager.cc:1939] T 7c070be3517043768198c581003c4e18 P c2ddc062f804456a94d49c91bc89ae76: Deleting consensus metadata
I20260809 06:25:14.162858 10447 catalog_manager.cc:4841] TS c2ddc062f804456a94d49c91bc89ae76 (127.8.131.2:40123): tablet 7c070be3517043768198c581003c4e18 (table test-table [id=648a91034b664ad2ae897433cf4c2c50]) successfully deleted
I20260809 06:25:14.168926 11317 ts_tablet_manager.cc:1918] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:25:14.169250 11317 log.cc:1196] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/7c070be3517043768198c581003c4e18
I20260809 06:25:14.169834 11317 ts_tablet_manager.cc:1939] T 7c070be3517043768198c581003c4e18 P 9ab31f6549c144c88366ffe8d27c68ff: Deleting consensus metadata
I20260809 06:25:14.171247 10450 catalog_manager.cc:4841] TS 9ab31f6549c144c88366ffe8d27c68ff (127.8.131.3:41779): tablet 7c070be3517043768198c581003c4e18 (table test-table [id=648a91034b664ad2ae897433cf4c2c50]) successfully deleted
I20260809 06:25:14.171839 11384 tablet_replica.cc:330] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: stopping tablet replica
I20260809 06:25:14.172149 11384 raft_consensus.cc:2229] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:25:14.172863 11384 raft_consensus.cc:2258] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:14.174103 11384 ts_tablet_manager.cc:1905] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:14.182250 11384 ts_tablet_manager.cc:1918] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:25:14.182512 11384 log.cc:1196] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/7c070be3517043768198c581003c4e18
I20260809 06:25:14.183130 11384 ts_tablet_manager.cc:1939] T 7c070be3517043768198c581003c4e18 P 15685ffdd66f4a19885ba3ad33719ed7: Deleting consensus metadata
I20260809 06:25:14.184741 10447 catalog_manager.cc:4841] TS 15685ffdd66f4a19885ba3ad33719ed7 (127.8.131.1:34753): tablet 7c070be3517043768198c581003c4e18 (table test-table [id=648a91034b664ad2ae897433cf4c2c50]) successfully deleted
Aug 09 06:25:14 dist-test-slave-mjjr krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:14.201962 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.172538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36886 (local address 127.8.131.1:34753)
0809 06:25:14.172745 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:14.172751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:14.180334 (+ 7583us) server_negotiation.cc:407] Connection header received
0809 06:25:14.180882 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:14.180889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:14.181000 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:14.181086 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:14.183322 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.184568 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.186937 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.187636 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.187805 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.193830 (+ 6025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:14.193847 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:14.193876 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:14.193939 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:14.198368 (+ 4429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.199254 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.199264 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.199268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.199451 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.199940 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.199945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.199949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.200466 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:14.200829 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:14.201176 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 06:25:14.201441 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 06:25:14.202817 10805 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36886
I20260809 06:25:14.229120 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.206772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42912 (local address 127.8.131.62:41235)
0809 06:25:14.206993 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:14.207001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:14.212186 (+ 5185us) server_negotiation.cc:407] Connection header received
0809 06:25:14.212388 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:14.212395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:14.212512 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:14.212635 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:14.214416 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.215700 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.217582 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.218337 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.218486 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.221570 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:14.221578 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:14.221584 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:14.221707 (+ 123us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:14.225519 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.226331 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.226339 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.226343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.226576 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.227066 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.227072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.227076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.227450 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:14.227777 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:14.228440 (+ 663us) server_negotiation.cc:299] Negotiation successful
0809 06:25:14.228691 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 06:25:14.229894 10477 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42912
I20260809 06:25:14.232784 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 10586
I20260809 06:25:14.251727 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.234353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.8.131.1:34753)
0809 06:25:14.234610 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:14.234617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:14.235350 (+ 733us) server_negotiation.cc:407] Connection header received
0809 06:25:14.235514 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:14.235520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:14.235605 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:14.235751 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:14.237191 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.238334 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.240218 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.240994 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.241131 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.244246 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:14.244254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:14.244260 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:14.244309 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:14.248492 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.249269 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.249275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.249280 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.249455 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.249886 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.249896 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.249900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.250275 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:14.250552 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:14.250961 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 06:25:14.251233 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260809 06:25:14.252501 10805 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36894
W20260809 06:25:14.253160 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 10586: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:25:14.253348 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 10586
I20260809 06:25:14.288061 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 10815
Aug 09 06:25:14 dist-test-slave-mjjr krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:25:14.308178 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.288966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.8.131.2:40123)
0809 06:25:14.289882 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:14.289889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:14.289911 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:25:14.289987 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:14.289993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:14.290081 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:14.290208 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:14.291785 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.292921 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.294732 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.295445 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.295594 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.301131 (+ 5537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:14.301149 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:14.301170 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:14.301216 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:14.305231 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.306140 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.306147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.306149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.306282 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.306709 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.306712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.306714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.307061 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:14.307284 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:14.307726 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 06:25:14.307908 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":617,"threads_started":1}
W20260809 06:25:14.308934 11015 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32820
W20260809 06:25:14.309585 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 10815: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:25:14.309770 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 10815
I20260809 06:25:14.334196 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 11029
Aug 09 06:25:14 dist-test-slave-mjjr krb5kdc[10404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:25:14.352211 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:14.335111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.8.131.3:41779)
0809 06:25:14.335304 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:14.335310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:14.335334 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:25:14.335473 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:14.335479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:14.335564 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:14.335706 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:14.337005 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.338198 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.339438 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:14.340043 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:14.340190 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:14.344867 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:14.344888 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:14.344912 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:14.344960 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:14.349115 (+ 4155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.349789 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.349798 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.349803 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.349988 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:14.350331 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:14.350337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:14.350341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:14.350861 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:14.351276 (+ 415us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:14.351452 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 06:25:14.351761 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 06:25:14.353008 11230 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53188
W20260809 06:25:14.353698 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 11029: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:25:14.353847 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 11029
I20260809 06:25:14.382103 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 10426
I20260809 06:25:14.383255 10538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:14.499359 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 10426
2026-08-09T06:25:14Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5123 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:14 dist-test-slave-mjjr krb5kdc[11399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:14 dist-test-slave-mjjr krb5kdc[11399](info): set up 2 sockets
Aug 09 06:25:14 dist-test-slave-mjjr krb5kdc[11399](info): commencing operation
krb5kdc: starting...
W20260809 06:25:15.810585 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:15 dist-test-slave-mjjr krb5kdc[11399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:16.015702 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:37599
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:37599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:16.341277 11421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:16.341585 11421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:16.341682 11421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:16.348816 11421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:16.348917 11421 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 06:25:16.348964 11421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:16.349025 11421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:16.349073 11421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:16.349135 11421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:16.358713 11421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:37599
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:37599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:16.359894 11421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:16.361150 11421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:16.372690 11431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:16.372646 11434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:16.372725 11432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:16.373852 11421 server_base.cc:1034] running on GCE node
I20260809 06:25:16.374617 11421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:16.376025 11421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:16.377264 11421 hybrid_clock.cc:648] HybridClock initialized: now 1786256716377209 us; error 33 us; skew 500 ppm
I20260809 06:25:16.377583 11421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:16 dist-test-slave-mjjr krb5kdc[11399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256716, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:16.380764 11421 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:16.382272 11421 webserver.cc:460] Webserver started at http://127.8.131.62:34715/ using document root <none> and password file <none>
I20260809 06:25:16.382948 11421 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:16.383075 11421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:16.383394 11421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:16.386395 11421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "2c8db8228820422e984f6d6645b0e4b3"
format_stamp: "Formatted at 2026-08-09 06:25:16 on dist-test-slave-mjjr"
server_key: "3bf86c07b7cc755170ce4a2d6e8ec93c"
server_key_iv: "9dfa8d1900787fdfb56c882d1cc6948d"
server_key_version: "encryptionkey@0"
I20260809 06:25:16.387158 11421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "2c8db8228820422e984f6d6645b0e4b3"
format_stamp: "Formatted at 2026-08-09 06:25:16 on dist-test-slave-mjjr"
server_key: "3bf86c07b7cc755170ce4a2d6e8ec93c"
server_key_iv: "9dfa8d1900787fdfb56c882d1cc6948d"
server_key_version: "encryptionkey@0"
I20260809 06:25:16.392103 11421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:25:16.395263 11447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:16.396582 11421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:16.396754 11421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "2c8db8228820422e984f6d6645b0e4b3"
format_stamp: "Formatted at 2026-08-09 06:25:16 on dist-test-slave-mjjr"
server_key: "3bf86c07b7cc755170ce4a2d6e8ec93c"
server_key_iv: "9dfa8d1900787fdfb56c882d1cc6948d"
server_key_version: "encryptionkey@0"
I20260809 06:25:16.396970 11421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:16.414433 11421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:16.418378 11421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:16.418639 11421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:16.440361 11421 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:37599
I20260809 06:25:16.440383 11555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:37599 every 8 connection(s)
I20260809 06:25:16.441702 11421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:16.445376 11556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:16.447196 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 11421
I20260809 06:25:16.447422 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:16.448172 8716 external_mini_cluster.cc:1428] Setting key 11d2462d9de65f7b5ae4600744a4e316
I20260809 06:25:16.458699 11556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3: Bootstrap starting.
I20260809 06:25:16.462456 11556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3: Neither blocks nor log segments found. Creating new log.
Aug 09 06:25:16 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:16.463527 11556 log.cc:824] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:16.466913 11556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3: No bootstrap required, opened a new log
I20260809 06:25:16.472177 11564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:16.450143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49394 (local address 127.8.131.62:37599)
0809 06:25:16.451021 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:16.451036 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:16.451094 (+ 58us) server_negotiation.cc:407] Connection header received
0809 06:25:16.453561 (+ 2467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:16.453589 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:16.454065 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:16.454514 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:16.455832 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:16.457837 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:16.459050 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:16.460105 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:16.460302 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:16.464317 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:16.464325 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:16.464339 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:16.464385 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:16.468229 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:16.468875 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:16.468881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:16.468887 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:16.469056 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:16.469435 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:16.469441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:16.469452 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:16.469980 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:16.470297 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:16.470732 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 06:25:16.471156 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":357,"threads_started":1}
I20260809 06:25:16.474778 11556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8db8228820422e984f6d6645b0e4b3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37599 } }
I20260809 06:25:16.475185 11556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:16.475298 11556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8db8228820422e984f6d6645b0e4b3, State: Initialized, Role: FOLLOWER
I20260809 06:25:16.475893 11556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8db8228820422e984f6d6645b0e4b3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37599 } }
I20260809 06:25:16.476132 11556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:16.476284 11556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:16.476442 11556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:16.478645 11556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8db8228820422e984f6d6645b0e4b3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37599 } }
I20260809 06:25:16.479178 11556 leader_election.cc:302] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c8db8228820422e984f6d6645b0e4b3; no voters:
I20260809 06:25:16.479878 11556 leader_election.cc:288] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:16.480005 11568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:16.480707 11568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8db8228820422e984f6d6645b0e4b3, State: Running, Role: LEADER
I20260809 06:25:16.481317 11568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8db8228820422e984f6d6645b0e4b3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37599 } }
I20260809 06:25:16.482240 11556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:16.484454 11570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c8db8228820422e984f6d6645b0e4b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8db8228820422e984f6d6645b0e4b3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37599 } } }
I20260809 06:25:16.484846 11570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:16.484233 11571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c8db8228820422e984f6d6645b0e4b3. Latest consensus state: current_term: 1 leader_uuid: "2c8db8228820422e984f6d6645b0e4b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8db8228820422e984f6d6645b0e4b3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37599 } } }
I20260809 06:25:16.485126 11571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:16.488179 11580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:16.493968 11580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:16.502562 11580 catalog_manager.cc:1347] Generated new cluster ID: 4b082a970bf54949b8241ab619f8df0e
I20260809 06:25:16.502699 11580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:16.514298 11580 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:16.515611 11580 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:16.526597 11580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3: Generated new TSK 0
I20260809 06:25:16.527365 11580 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:25:16.572407 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:37599
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:16.904892 11606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:16.905169 11606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:16.905355 11606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:16.912581 11606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:16.912705 11606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:16.912855 11606 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 06:25:16.912933 11606 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 06:25:16.913059 11606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:16.922974 11606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.131.62:37599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:16.924330 11606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:16.925601 11606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:16.938715 11613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:16.938741 11620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:16.938741 11614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:16.939946 11606 server_base.cc:1034] running on GCE node
I20260809 06:25:16.940517 11606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:16.941375 11606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:16.942585 11606 hybrid_clock.cc:648] HybridClock initialized: now 1786256716942551 us; error 50 us; skew 500 ppm
I20260809 06:25:16.942866 11606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:16 dist-test-slave-mjjr krb5kdc[11399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256716, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:16.949615 11606 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:25:16.951053 11606 webserver.cc:460] Webserver started at http://127.8.131.1:46323/ using document root <none> and password file <none>
I20260809 06:25:16.951785 11606 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:16.951928 11606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:16.952240 11606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:16.955062 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "14ee81a1a49a411586a8bd73c4555613"
format_stamp: "Formatted at 2026-08-09 06:25:16 on dist-test-slave-mjjr"
server_key: "99e92c7a5b2c35009df7b740c7bb0d51"
server_key_iv: "a7dab89019ca84f8db9b52c9c17b4dbc"
server_key_version: "encryptionkey@0"
I20260809 06:25:16.955912 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "14ee81a1a49a411586a8bd73c4555613"
format_stamp: "Formatted at 2026-08-09 06:25:16 on dist-test-slave-mjjr"
server_key: "99e92c7a5b2c35009df7b740c7bb0d51"
server_key_iv: "a7dab89019ca84f8db9b52c9c17b4dbc"
server_key_version: "encryptionkey@0"
I20260809 06:25:16.960948 11606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:25:16.964421 11630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:16.965801 11606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 06:25:16.965981 11606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "14ee81a1a49a411586a8bd73c4555613"
format_stamp: "Formatted at 2026-08-09 06:25:16 on dist-test-slave-mjjr"
server_key: "99e92c7a5b2c35009df7b740c7bb0d51"
server_key_iv: "a7dab89019ca84f8db9b52c9c17b4dbc"
server_key_version: "encryptionkey@0"
I20260809 06:25:16.966194 11606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:16.988137 11606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:16.992180 11606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:16.992483 11606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:16.995163 11606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:16.995290 11606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:16.995412 11606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:16.995543 11606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 06:25:17 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256716, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:17.020411 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:16.995836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:37389 (local address 127.8.131.62:37599)
0809 06:25:16.998276 (+ 2440us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:16.998285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:16.998312 (+ 27us) server_negotiation.cc:407] Connection header received
0809 06:25:17.000296 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:17.000302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:17.000399 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:17.000517 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:17.002733 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.003853 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.006375 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.006824 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.006967 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:17.012176 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:17.012185 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:17.012192 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:17.012241 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:17.015323 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:17.016471 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:17.016480 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:17.016483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:17.016641 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:17.017232 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:17.017240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:17.017245 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:17.017660 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:17.017846 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:17.019844 (+ 1998us) server_negotiation.cc:299] Negotiation successful
0809 06:25:17.020065 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2304,"thread_start_us":2184,"threads_started":1}
I20260809 06:25:17.020488 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:16.996506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37599 (local address 127.8.131.1:37389)
0809 06:25:16.997636 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:16.997703 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:16.999954 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:17.000806 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:17.000818 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:17.001599 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:17.002518 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:17.002553 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.003999 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.004004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:17.006217 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:17.006229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.007079 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.007089 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:17.007312 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:17.008079 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:17.008101 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:17.011892 (+ 3791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:17.015466 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:17.015480 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:17.015501 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:17.016321 (+ 820us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:17.016761 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:17.016767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:17.016771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:17.017075 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:17.017858 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:17.017872 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:17.018341 (+ 469us) client_negotiation.cc:769] Sending connection context
0809 06:25:17.018708 (+ 367us) client_negotiation.cc:240] Negotiation successful
0809 06:25:17.019123 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":452,"threads_started":1}
I20260809 06:25:17.048509 11606 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:43385
I20260809 06:25:17.048597 11833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:43385 every 8 connection(s)
I20260809 06:25:17.049420 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:17.032348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37599 (local address 127.8.131.1:44575)
0809 06:25:17.033205 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:17.033230 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:17.033401 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:17.034090 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:17.034096 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:17.034894 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:17.035488 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:17.035509 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.037168 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.037176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:17.039164 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:17.039181 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.040273 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.040284 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:17.040521 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:17.041297 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:17.041314 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:17.042031 (+ 717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:17.046386 (+ 4355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:17.046392 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:17.046397 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:17.046856 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:17.047315 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:17.047322 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:17.047325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:17.047507 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:17.048254 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:17.048261 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:17.048537 (+ 276us) client_negotiation.cc:769] Sending connection context
0809 06:25:17.048753 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 06:25:17.049047 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":452,"threads_started":1}
I20260809 06:25:17.049459 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:17.031897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:44575 (local address 127.8.131.62:37599)
0809 06:25:17.032120 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:17.032129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:17.033334 (+ 1205us) server_negotiation.cc:407] Connection header received
0809 06:25:17.033618 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:17.033624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:17.033721 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:17.033804 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:17.035729 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.037010 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.039320 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.040057 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.040219 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:17.042246 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:17.042251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:17.042256 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:17.042295 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:17.046231 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:17.046978 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:17.046987 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:17.046990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:17.047163 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:17.047670 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:17.047679 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:17.047685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:17.048107 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:17.048266 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:17.048873 (+ 607us) server_negotiation.cc:299] Negotiation successful
0809 06:25:17.049100 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 06:25:17.050210 11606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:17.052390 11834 heartbeater.cc:347] Connected to a master server at 127.8.131.62:37599
I20260809 06:25:17.052649 11834 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:17.053308 11834 heartbeater.cc:511] Master 127.8.131.62:37599 requested a full tablet report, sending...
I20260809 06:25:17.054204 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 11606
I20260809 06:25:17.054409 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:17.055065 8716 external_mini_cluster.cc:1428] Setting key b3c3065071061f2ab7dd9d6aed91277b
I20260809 06:25:17.055297 11472 ts_manager.cc:194] Registered new tserver with Master: 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385)
I20260809 06:25:17.057008 11472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:37389
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:25:17.098717 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:37599
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:17.420786 11840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:17.421113 11840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:17.421285 11840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:17.428709 11840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:17.428862 11840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:17.429008 11840 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 06:25:17.429095 11840 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 06:25:17.429221 11840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:17.439028 11840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.131.62:37599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:17.440466 11840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:17.441771 11840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:17.455204 11848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:17.455229 11850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:17.455806 11846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:17.456195 11840 server_base.cc:1034] running on GCE node
I20260809 06:25:17.456794 11840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:17.457702 11840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:17.458930 11840 hybrid_clock.cc:648] HybridClock initialized: now 1786256717458896 us; error 76 us; skew 500 ppm
I20260809 06:25:17.459235 11840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:17 dist-test-slave-mjjr krb5kdc[11399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256717, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:17.462325 11840 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:25:17.464476 11840 webserver.cc:460] Webserver started at http://127.8.131.2:37935/ using document root <none> and password file <none>
I20260809 06:25:17.465227 11840 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:17.465365 11840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:17.465705 11840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:17.468720 11840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "215626758e634a91bd415d06711d8533"
format_stamp: "Formatted at 2026-08-09 06:25:17 on dist-test-slave-mjjr"
server_key: "bfd023dd1516c4931bdd754be114a0ea"
server_key_iv: "3b09f5f515679430174dea926d3dff22"
server_key_version: "encryptionkey@0"
I20260809 06:25:17.469552 11840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "215626758e634a91bd415d06711d8533"
format_stamp: "Formatted at 2026-08-09 06:25:17 on dist-test-slave-mjjr"
server_key: "bfd023dd1516c4931bdd754be114a0ea"
server_key_iv: "3b09f5f515679430174dea926d3dff22"
server_key_version: "encryptionkey@0"
I20260809 06:25:17.474733 11840 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 06:25:17.478081 11858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:17.479331 11840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:25:17.479468 11840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "215626758e634a91bd415d06711d8533"
format_stamp: "Formatted at 2026-08-09 06:25:17 on dist-test-slave-mjjr"
server_key: "bfd023dd1516c4931bdd754be114a0ea"
server_key_iv: "3b09f5f515679430174dea926d3dff22"
server_key_version: "encryptionkey@0"
I20260809 06:25:17.479629 11840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:17.500979 11840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:17.505093 11840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:17.505415 11840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:17.508036 11840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:17.508164 11840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:17.508285 11840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:17.508375 11840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 06:25:17 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256717, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:17.532547 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:17.508708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:39959 (local address 127.8.131.62:37599)
0809 06:25:17.508899 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:17.508908 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:17.510543 (+ 1635us) server_negotiation.cc:407] Connection header received
0809 06:25:17.512736 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:17.512744 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:17.512874 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:17.512954 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:17.515684 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.516870 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.519546 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.520253 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.520403 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:17.525198 (+ 4795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:17.525210 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:17.525213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:17.525248 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:17.528889 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:17.530056 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:17.530060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:17.530063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:17.530187 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:17.530654 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:17.530660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:17.530663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:17.530965 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:17.531128 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:17.531947 (+ 819us) server_negotiation.cc:299] Negotiation successful
0809 06:25:17.532195 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 06:25:17.534271 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:17.509445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37599 (local address 127.8.131.2:39959)
0809 06:25:17.510377 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:17.510423 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:17.512353 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:17.513720 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:17.513771 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:17.514611 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:17.515437 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:17.515464 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.517064 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.517071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:17.519382 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:17.519402 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.520546 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.520554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:17.520847 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:17.521818 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:17.521868 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:17.524955 (+ 3087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:17.529118 (+ 4163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:17.529133 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:17.529153 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:17.529891 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:17.530322 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:17.530328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:17.530332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:17.530534 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:17.531119 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:17.531127 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:17.531505 (+ 378us) client_negotiation.cc:769] Sending connection context
0809 06:25:17.531858 (+ 353us) client_negotiation.cc:240] Negotiation successful
0809 06:25:17.532479 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":469,"threads_started":1}
I20260809 06:25:17.554380 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:17.540534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37599 (local address 127.8.131.2:59721)
0809 06:25:17.541396 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:17.541424 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:17.541596 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:17.541972 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:17.541977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:17.542447 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:17.542769 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:17.542779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.544160 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.544166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:17.545174 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:17.545186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.545935 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.545943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:17.546112 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:17.546881 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:17.546898 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:17.548050 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:17.551360 (+ 3310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:17.551366 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:17.551370 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:17.551949 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:17.552441 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:17.552444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:17.552446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:17.552563 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:17.553351 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:17.553354 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:17.553502 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 06:25:17.553614 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 06:25:17.553936 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":489,"threads_started":1}
I20260809 06:25:17.554549 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:17.540459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:59721 (local address 127.8.131.62:37599)
0809 06:25:17.540661 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:17.540666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:17.541536 (+ 870us) server_negotiation.cc:407] Connection header received
0809 06:25:17.541757 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:17.541763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:17.541840 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:17.541939 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:17.542922 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.544020 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.545316 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:17.545742 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:17.545870 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:17.548269 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:17.548278 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:17.548285 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:17.548331 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:17.551213 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:17.552087 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:17.552096 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:17.552102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:17.552312 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:17.552670 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:17.552678 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:17.552684 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:17.553226 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:17.553420 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:17.553762 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 06:25:17.554068 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 06:25:17.569983 11840 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:41839
I20260809 06:25:17.570020 12049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:41839 every 8 connection(s)
I20260809 06:25:17.571244 11840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:17.573378 12050 heartbeater.cc:347] Connected to a master server at 127.8.131.62:37599
I20260809 06:25:17.573607 12050 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:17.574275 12050 heartbeater.cc:511] Master 127.8.131.62:37599 requested a full tablet report, sending...
I20260809 06:25:17.575568 11472 ts_manager.cc:194] Registered new tserver with Master: 215626758e634a91bd415d06711d8533 (127.8.131.2:41839)
I20260809 06:25:17.576452 11472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:39959
I20260809 06:25:17.577971 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 11840
I20260809 06:25:17.578153 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:17.578616 8716 external_mini_cluster.cc:1428] Setting key 95fa09f73f3ceeb931f75f61cb3e8ac0
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:25:17.622901 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:37599
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:17.950642 12055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:17.950950 12055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:17.951145 12055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:17.958379 12055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:17.958492 12055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:17.958612 12055 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 06:25:17.958710 12055 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 06:25:17.958863 12055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:17.968622 12055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.131.62:37599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:17.970149 12055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:17.971535 12055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:17.985373 12065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:17.985275 12069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:17.985255 12067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:17.986886 12055 server_base.cc:1034] running on GCE node
I20260809 06:25:17.987489 12055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:17.988508 12055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:17.989743 12055 hybrid_clock.cc:648] HybridClock initialized: now 1786256717989700 us; error 36 us; skew 500 ppm
I20260809 06:25:17.990033 12055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:17 dist-test-slave-mjjr krb5kdc[11399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256717, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:17.993227 12055 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:25:17.994626 12055 webserver.cc:460] Webserver started at http://127.8.131.3:44921/ using document root <none> and password file <none>
I20260809 06:25:17.995292 12055 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:17.995399 12055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:17.995730 12055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:17.998621 12055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "2a4145153bd24ab0beacacd1e288ceea"
format_stamp: "Formatted at 2026-08-09 06:25:17 on dist-test-slave-mjjr"
server_key: "e872259546be2191c2179e87514f0725"
server_key_iv: "d97b096ee432a05b330038bc172673c1"
server_key_version: "encryptionkey@0"
I20260809 06:25:17.999446 12055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "2a4145153bd24ab0beacacd1e288ceea"
format_stamp: "Formatted at 2026-08-09 06:25:17 on dist-test-slave-mjjr"
server_key: "e872259546be2191c2179e87514f0725"
server_key_iv: "d97b096ee432a05b330038bc172673c1"
server_key_version: "encryptionkey@0"
I20260809 06:25:18.004909 12055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:25:18.008381 12083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:18.009836 12055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:18.009975 12055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "2a4145153bd24ab0beacacd1e288ceea"
format_stamp: "Formatted at 2026-08-09 06:25:17 on dist-test-slave-mjjr"
server_key: "e872259546be2191c2179e87514f0725"
server_key_iv: "d97b096ee432a05b330038bc172673c1"
server_key_version: "encryptionkey@0"
I20260809 06:25:18.010142 12055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:18.030519 12055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:18.035127 12055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:18.035386 12055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:18.038329 12055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:18.038429 12055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:18.038640 12055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:18.038800 12055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 06:25:18 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256717, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:18.059531 11834 heartbeater.cc:503] Master 127.8.131.62:37599 was elected leader, sending a full tablet report...
I20260809 06:25:18.064337 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.038904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:60071 (local address 127.8.131.62:37599)
0809 06:25:18.039116 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.039123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.040405 (+ 1282us) server_negotiation.cc:407] Connection header received
0809 06:25:18.042508 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.042514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.042606 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.042713 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:18.045078 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.046174 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.048878 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.049831 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.050006 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.056003 (+ 5997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:18.056012 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:18.056017 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:18.056063 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:18.060393 (+ 4330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:18.061396 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:18.061401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:18.061405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:18.061598 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:18.062206 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:18.062210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:18.062212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:18.062504 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:18.062617 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.063785 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.064067 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 06:25:18.066172 12101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.039299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37599 (local address 127.8.131.3:60071)
0809 06:25:18.040223 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.040279 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.042198 (+ 1919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.042961 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.042969 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.043809 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:18.044814 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.044846 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.046339 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.046347 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.048700 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.048716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.050432 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.050443 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.050804 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.051737 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:18.051789 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:18.055678 (+ 3889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:18.060626 (+ 4948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:18.060635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:18.060649 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:18.061262 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:18.061739 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:18.061747 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:18.061754 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:18.062078 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:18.062635 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:18.062646 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:18.063187 (+ 541us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.063581 (+ 394us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.064308 (+ 727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":426,"threads_started":1}
I20260809 06:25:18.095451 12055 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:38077
I20260809 06:25:18.095453 12281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:38077 every 8 connection(s)
I20260809 06:25:18.096241 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.077541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37599 (local address 127.8.131.3:42149)
0809 06:25:18.079031 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.079058 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.079231 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.079916 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.079919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.080874 (+ 955us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:18.081435 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.081461 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.083143 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.083152 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.084500 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.084517 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.085908 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.085915 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.086157 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.086868 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:18.086893 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:18.088354 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:18.093043 (+ 4689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:18.093049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:18.093054 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:18.093546 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:18.093978 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:18.093982 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:18.093984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:18.094112 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:18.094949 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:18.094958 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:18.095200 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.095403 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.095800 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1271,"thread_start_us":927,"threads_started":1}
I20260809 06:25:18.097193 12055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:18.097342 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.077847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:42149 (local address 127.8.131.62:37599)
0809 06:25:18.078081 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.078091 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.079165 (+ 1074us) server_negotiation.cc:407] Connection header received
0809 06:25:18.079428 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.079436 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.079574 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.079681 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:18.081660 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.082966 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.084661 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.085450 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.085613 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.088661 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:18.088671 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:18.088679 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:18.088753 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:18.092875 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:18.093679 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:18.093684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:18.093687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:18.093846 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:18.094255 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:18.094264 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:18.094281 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:18.094692 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:18.094859 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.096724 (+ 1865us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.096950 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 06:25:18.099551 12282 heartbeater.cc:347] Connected to a master server at 127.8.131.62:37599
I20260809 06:25:18.100051 12282 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:18.101054 12282 heartbeater.cc:511] Master 127.8.131.62:37599 requested a full tablet report, sending...
I20260809 06:25:18.101464 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 12055
I20260809 06:25:18.101678 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:18.102138 8716 external_mini_cluster.cc:1428] Setting key c2580fbf6c940bbbe83db4ad7b652d0f
I20260809 06:25:18.102530 11472 ts_manager.cc:194] Registered new tserver with Master: 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077)
I20260809 06:25:18.103600 11472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:60071
I20260809 06:25:18.105049 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 06:25:18 dist-test-slave-mjjr krb5kdc[11399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 06:25:18 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:18.155042 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.140752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49404 (local address 127.8.131.62:37599)
0809 06:25:18.141010 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.141017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.141187 (+ 170us) server_negotiation.cc:407] Connection header received
0809 06:25:18.141408 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.141412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.141482 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.141586 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:18.142958 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.143751 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.144962 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.145393 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.145524 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.149481 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:18.149487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:18.149490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:18.149528 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:18.152206 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:18.152881 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:18.152892 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:18.152895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:18.153030 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:18.153444 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:18.153451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:18.153455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:18.153788 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:18.154056 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.154400 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.154644 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 06:25:18.159057 11472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:25:18.166035 11472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:18.199853 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.178577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38512 (local address 127.8.131.1:43385)
0809 06:25:18.191683 (+ 13106us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.191695 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.191732 (+ 37us) server_negotiation.cc:407] Connection header received
0809 06:25:18.191856 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.191868 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.192176 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.192414 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:18.193188 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.194565 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.197514 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.198618 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.198775 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.199037 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.199320 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.199473 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11529,"thread_start_us":13580,"threads_started":1}
I20260809 06:25:18.200001 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.178302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43385 (local address 127.0.0.1:38512)
0809 06:25:18.188836 (+ 10534us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.188898 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.189116 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.192421 (+ 3305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.192429 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.192456 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:18.192906 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.192928 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.194712 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.194717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.197346 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.197364 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.198830 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.198836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.199052 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.199114 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.199257 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.199444 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10173,"mutex_wait_us":6,"thread_start_us":13072,"threads_started":1}
I20260809 06:25:18.200480 12313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.189420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:41839 (local address 127.0.0.1:47642)
0809 06:25:18.191948 (+ 2528us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.191974 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.192067 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.194001 (+ 1934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.194006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.194032 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:18.194352 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.194363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.195814 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.195819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.197784 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.197799 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.199551 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.199557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.199798 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.199844 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.199916 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.200041 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2413,"mutex_wait_us":43,"spinlock_wait_cycles":4480,"thread_start_us":2281,"threads_started":1}
I20260809 06:25:18.200677 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.178491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:38077 (local address 127.0.0.1:37988)
0809 06:25:18.191900 (+ 13409us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.191917 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.192067 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.192793 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.192801 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.192840 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:18.193345 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.193359 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.195295 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.195300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.197638 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.197648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.199859 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.199864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.200035 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.200063 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.200144 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.200278 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13036,"mutex_wait_us":104,"spinlock_wait_cycles":384,"thread_start_us":9836,"threads_started":1}
I20260809 06:25:18.201107 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.179447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.8.131.3:38077)
0809 06:25:18.190065 (+ 10618us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.190079 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.192191 (+ 2112us) server_negotiation.cc:407] Connection header received
0809 06:25:18.192295 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.192302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.192560 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.192768 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:18.193547 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.195127 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.197782 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.199314 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.199451 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.199712 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.200609 (+ 897us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.200761 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10366,"thread_start_us":10149,"threads_started":1}
I20260809 06:25:18.202841 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.189668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47642 (local address 127.8.131.2:41839)
0809 06:25:18.193473 (+ 3805us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.193481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.193507 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:25:18.193606 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.193612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.193802 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.194000 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:18.194539 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.195631 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.197954 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.199316 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.202106 (+ 2790us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.202323 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.202405 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.202562 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3623,"thread_start_us":3511,"threads_started":1}
I20260809 06:25:18.203936 11719 tablet_service.cc:1467] Processing CreateTablet for tablet 7c63095f35544854a2d9e325c424399a (DEFAULT_TABLE table=test-table [id=94e931b7f77e4c6d9a291518102ecfba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:18.204444 12164 tablet_service.cc:1467] Processing CreateTablet for tablet 7c63095f35544854a2d9e325c424399a (DEFAULT_TABLE table=test-table [id=94e931b7f77e4c6d9a291518102ecfba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:18.205534 11719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c63095f35544854a2d9e325c424399a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:18.205564 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c63095f35544854a2d9e325c424399a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:18.206328 11941 tablet_service.cc:1467] Processing CreateTablet for tablet 7c63095f35544854a2d9e325c424399a (DEFAULT_TABLE table=test-table [id=94e931b7f77e4c6d9a291518102ecfba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:18.207604 11941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c63095f35544854a2d9e325c424399a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:18.216743 12318 tablet_bootstrap.cc:492] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: Bootstrap starting.
I20260809 06:25:18.220602 12320 tablet_bootstrap.cc:492] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: Bootstrap starting.
I20260809 06:25:18.221019 12318 tablet_bootstrap.cc:654] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:18.222431 12318 log.cc:824] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:18.224270 12320 tablet_bootstrap.cc:654] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:18.225610 12320 log.cc:824] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:18.225817 12318 tablet_bootstrap.cc:492] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: No bootstrap required, opened a new log
I20260809 06:25:18.225905 12321 tablet_bootstrap.cc:492] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: Bootstrap starting.
I20260809 06:25:18.226116 12318 ts_tablet_manager.cc:1397] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260809 06:25:18.228638 12320 tablet_bootstrap.cc:492] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: No bootstrap required, opened a new log
I20260809 06:25:18.228720 12321 tablet_bootstrap.cc:654] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:18.228963 12320 ts_tablet_manager.cc:1397] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260809 06:25:18.229943 12321 log.cc:824] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:18.232261 12321 tablet_bootstrap.cc:492] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: No bootstrap required, opened a new log
I20260809 06:25:18.232504 12321 ts_tablet_manager.cc:1397] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 06:25:18.236933 12318 raft_consensus.cc:348] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } }
I20260809 06:25:18.237413 12318 raft_consensus.cc:374] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:18.237516 12318 raft_consensus.cc:729] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4145153bd24ab0beacacd1e288ceea, State: Initialized, Role: FOLLOWER
I20260809 06:25:18.238150 12318 consensus_queue.cc:260] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } }
I20260809 06:25:18.239334 12282 heartbeater.cc:503] Master 127.8.131.62:37599 was elected leader, sending a full tablet report...
I20260809 06:25:18.239670 12320 raft_consensus.cc:348] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } }
I20260809 06:25:18.240068 12318 ts_tablet_manager.cc:1428] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 06:25:18.240114 12320 raft_consensus.cc:374] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:18.240214 12320 raft_consensus.cc:729] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14ee81a1a49a411586a8bd73c4555613, State: Initialized, Role: FOLLOWER
I20260809 06:25:18.240521 12321 raft_consensus.cc:348] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } }
I20260809 06:25:18.240908 12321 raft_consensus.cc:374] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:18.241015 12321 raft_consensus.cc:729] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215626758e634a91bd415d06711d8533, State: Initialized, Role: FOLLOWER
I20260809 06:25:18.240901 12320 consensus_queue.cc:260] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } }
I20260809 06:25:18.241636 12321 consensus_queue.cc:260] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } }
I20260809 06:25:18.242700 12320 ts_tablet_manager.cc:1428] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
I20260809 06:25:18.244098 12050 heartbeater.cc:503] Master 127.8.131.62:37599 was elected leader, sending a full tablet report...
I20260809 06:25:18.245082 12321 ts_tablet_manager.cc:1428] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
W20260809 06:25:18.302644 11837 tablet.cc:2369] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:25:18.323314 12051 tablet.cc:2369] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:25:18.349118 12283 tablet.cc:2369] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:25:18.441779 12325 raft_consensus.cc:482] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:18.442112 12325 raft_consensus.cc:504] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } }
I20260809 06:25:18.444222 12325 leader_election.cc:288] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385), 215626758e634a91bd415d06711d8533 (127.8.131.2:41839)
I20260809 06:25:18.450829 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.444706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:48963 (local address 127.8.131.2:41839)
0809 06:25:18.444918 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.444924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.445087 (+ 163us) server_negotiation.cc:407] Connection header received
0809 06:25:18.445294 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.445300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.445379 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.445474 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:18.446078 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.447248 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.449090 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.449901 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.450041 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.450148 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.450445 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.450578 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:25:18.450835 12101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.444661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:41839 (local address 127.8.131.3:48963)
0809 06:25:18.444967 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.444989 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.445152 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.445654 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.445660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.445690 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:18.445917 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.445927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.447392 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.447400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.448928 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.448940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.450061 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.450067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.450255 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.450279 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.450410 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.450550 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":1}
I20260809 06:25:18.452164 11975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c63095f35544854a2d9e325c424399a" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533" is_pre_election: true
I20260809 06:25:18.452597 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.444725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:42589 (local address 127.8.131.1:43385)
0809 06:25:18.444969 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.444975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.446029 (+ 1054us) server_negotiation.cc:407] Connection header received
0809 06:25:18.446222 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.446226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.446308 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.446424 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:18.446934 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.447871 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.450386 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.451629 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.451809 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.451916 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.452227 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.452363 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 06:25:18.452644 12331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.445185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43385 (local address 127.8.131.3:42589)
0809 06:25:18.445921 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.445936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.446075 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.446444 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.446448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.446491 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:18.446780 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.446788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.448011 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.448016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.450212 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.450226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.451866 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.451872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.452044 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.452077 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.452170 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.452301 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":42,"thread_start_us":406,"threads_started":1}
I20260809 06:25:18.452742 11975 raft_consensus.cc:2454] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 0.
I20260809 06:25:18.453526 12086 leader_election.cc:302] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215626758e634a91bd415d06711d8533, 2a4145153bd24ab0beacacd1e288ceea; no voters:
I20260809 06:25:18.453954 12325 raft_consensus.cc:2781] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:18.453823 11760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c63095f35544854a2d9e325c424399a" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14ee81a1a49a411586a8bd73c4555613" is_pre_election: true
I20260809 06:25:18.454110 12325 raft_consensus.cc:482] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:18.454201 12325 raft_consensus.cc:3037] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:18.454334 11760 raft_consensus.cc:2454] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 0.
I20260809 06:25:18.457432 12325 raft_consensus.cc:504] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } }
I20260809 06:25:18.458076 12325 leader_election.cc:288] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 election: Requested vote from peers 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385), 215626758e634a91bd415d06711d8533 (127.8.131.2:41839)
I20260809 06:25:18.458498 11975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c63095f35544854a2d9e325c424399a" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533"
I20260809 06:25:18.458487 11760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c63095f35544854a2d9e325c424399a" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14ee81a1a49a411586a8bd73c4555613"
I20260809 06:25:18.458670 11760 raft_consensus.cc:3037] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:18.458696 11975 raft_consensus.cc:3037] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:18.460942 11975 raft_consensus.cc:2454] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 1.
I20260809 06:25:18.461333 12086 leader_election.cc:302] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215626758e634a91bd415d06711d8533, 2a4145153bd24ab0beacacd1e288ceea; no voters:
I20260809 06:25:18.461621 12325 raft_consensus.cc:2781] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:18.461611 11760 raft_consensus.cc:2454] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 1.
I20260809 06:25:18.462137 12325 raft_consensus.cc:686] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 1 LEADER]: Becoming Leader. State: Replica: 2a4145153bd24ab0beacacd1e288ceea, State: Running, Role: LEADER
I20260809 06:25:18.462630 12325 consensus_queue.cc:237] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } }
I20260809 06:25:18.466923 11470 catalog_manager.cc:5495] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3). New cstate: current_term: 1 leader_uuid: "2a4145153bd24ab0beacacd1e288ceea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } health_report { overall_health: UNKNOWN } } }
Aug 09 06:25:18 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256716, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:18.524564 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.509998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.8.131.62:37599)
0809 06:25:18.510161 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.510165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.510709 (+ 544us) server_negotiation.cc:407] Connection header received
0809 06:25:18.510923 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.510927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.511001 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.511103 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:18.512346 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.513069 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.514338 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.514750 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.514891 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.518893 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:18.518901 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:18.518907 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:18.518947 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:18.521706 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:18.522512 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:18.522516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:18.522518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:18.522632 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:18.523091 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:18.523095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:18.523097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:18.523386 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:18.523524 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.524082 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.524310 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":51}
I20260809 06:25:18.524608 12342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.510035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37599 (local address 127.0.0.1:49408)
0809 06:25:18.510614 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.510629 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.510788 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.511125 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.511129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.511936 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:18.512210 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.512220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.513189 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.513197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.514199 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.514211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.514931 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.514936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.515080 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.515700 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:18.515717 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:18.518703 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:18.521851 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:18.521863 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:18.521869 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:18.522398 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:18.522757 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:18.522761 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:18.522763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:18.522913 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:18.523533 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:18.523539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:18.523784 (+ 245us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.523940 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.524224 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":348,"threads_started":1}
I20260809 06:25:18.529623 11470 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:49408:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 06:25:18.530700 11470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:18.537287 11719 tablet_service.cc:1467] Processing CreateTablet for tablet bb89440e231a4cd190da263582a33e55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66819d67973d46a09e37d79f4729a4af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 06:25:18.537424 11941 tablet_service.cc:1467] Processing CreateTablet for tablet bb89440e231a4cd190da263582a33e55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66819d67973d46a09e37d79f4729a4af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 06:25:18.537756 12164 tablet_service.cc:1467] Processing CreateTablet for tablet bb89440e231a4cd190da263582a33e55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66819d67973d46a09e37d79f4729a4af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 06:25:18.538348 11719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb89440e231a4cd190da263582a33e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:18.538357 11941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb89440e231a4cd190da263582a33e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:18.538560 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb89440e231a4cd190da263582a33e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:18.544119 12320 tablet_bootstrap.cc:492] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: Bootstrap starting.
I20260809 06:25:18.545361 12320 tablet_bootstrap.cc:654] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:18.545651 12321 tablet_bootstrap.cc:492] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: Bootstrap starting.
I20260809 06:25:18.545676 12318 tablet_bootstrap.cc:492] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: Bootstrap starting.
I20260809 06:25:18.547415 12318 tablet_bootstrap.cc:654] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:18.547803 12321 tablet_bootstrap.cc:654] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:18.549208 12320 tablet_bootstrap.cc:492] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: No bootstrap required, opened a new log
I20260809 06:25:18.549371 12320 ts_tablet_manager.cc:1397] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:25:18.549754 12318 tablet_bootstrap.cc:492] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: No bootstrap required, opened a new log
I20260809 06:25:18.549942 12318 ts_tablet_manager.cc:1397] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260809 06:25:18.550124 12320 raft_consensus.cc:348] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.550386 12320 raft_consensus.cc:374] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:18.550495 12321 tablet_bootstrap.cc:492] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: No bootstrap required, opened a new log
I20260809 06:25:18.550493 12320 raft_consensus.cc:729] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14ee81a1a49a411586a8bd73c4555613, State: Initialized, Role: FOLLOWER
I20260809 06:25:18.550617 12321 ts_tablet_manager.cc:1397] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 06:25:18.550732 12318 raft_consensus.cc:348] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.550772 12320 consensus_queue.cc:260] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.551424 12320 ts_tablet_manager.cc:1428] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 06:25:18.551383 12321 raft_consensus.cc:348] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.551307 12327 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.550971 12318 raft_consensus.cc:374] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:18.551630 12321 raft_consensus.cc:374] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:18.551780 12327 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:18.551769 12321 raft_consensus.cc:729] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215626758e634a91bd415d06711d8533, State: Initialized, Role: FOLLOWER
I20260809 06:25:18.551769 12318 raft_consensus.cc:729] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4145153bd24ab0beacacd1e288ceea, State: Initialized, Role: FOLLOWER
I20260809 06:25:18.551986 12321 consensus_queue.cc:260] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.552052 12318 consensus_queue.cc:260] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.552618 12321 ts_tablet_manager.cc:1428] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 06:25:18.552491 12328 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.552759 12318 ts_tablet_manager.cc:1428] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 06:25:18.552889 12328 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:18.552752 12325 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.553032 12325 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:18.599318 12327 raft_consensus.cc:482] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:18.599627 12327 raft_consensus.cc:504] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.601270 12327 leader_election.cc:288] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 215626758e634a91bd415d06711d8533 (127.8.131.2:41839), 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077)
I20260809 06:25:18.610271 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.601898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:44979 (local address 127.8.131.2:41839)
0809 06:25:18.602120 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.602127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.602595 (+ 468us) server_negotiation.cc:407] Connection header received
0809 06:25:18.602795 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.602801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.602927 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.603035 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:18.603850 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.604946 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.607724 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.608954 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.609073 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.609178 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.609857 (+ 679us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.609988 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:25:18.610258 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.602786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:46159 (local address 127.8.131.3:38077)
0809 06:25:18.603014 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.603021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.603412 (+ 391us) server_negotiation.cc:407] Connection header received
0809 06:25:18.603573 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.603577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.603633 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.603763 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:18.604234 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.605200 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.607721 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.608946 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.609074 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.609178 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.609857 (+ 679us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.609988 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 06:25:18.610456 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.602615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:38077 (local address 127.8.131.1:46159)
0809 06:25:18.603299 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.603322 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.603451 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.603785 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.603793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.603829 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:18.604089 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.604096 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.605336 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.605340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.607534 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.607549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.609278 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.609283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.609605 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.609719 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.609834 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.609983 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":154,"thread_start_us":302,"threads_started":1}
I20260809 06:25:18.610478 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.601764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:41839 (local address 127.8.131.1:44979)
0809 06:25:18.602460 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.602476 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.602643 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.603123 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.603128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.603155 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:18.603625 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.603635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.605073 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.605077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.607535 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.607549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.609261 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.609270 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.609608 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.609715 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.609814 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.609983 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":211,"thread_start_us":449,"threads_started":1}
I20260809 06:25:18.611279 11975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb89440e231a4cd190da263582a33e55" candidate_uuid: "14ee81a1a49a411586a8bd73c4555613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533" is_pre_election: true
I20260809 06:25:18.611372 12201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb89440e231a4cd190da263582a33e55" candidate_uuid: "14ee81a1a49a411586a8bd73c4555613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4145153bd24ab0beacacd1e288ceea" is_pre_election: true
I20260809 06:25:18.611562 11975 raft_consensus.cc:2454] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14ee81a1a49a411586a8bd73c4555613 in term 0.
I20260809 06:25:18.611778 12201 raft_consensus.cc:2454] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14ee81a1a49a411586a8bd73c4555613 in term 0.
I20260809 06:25:18.612067 11633 leader_election.cc:302] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14ee81a1a49a411586a8bd73c4555613, 215626758e634a91bd415d06711d8533; no voters:
I20260809 06:25:18.612433 12327 raft_consensus.cc:2781] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:18.612562 12327 raft_consensus.cc:482] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:18.612669 12327 raft_consensus.cc:3037] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:18.614920 12327 raft_consensus.cc:504] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.615561 12327 leader_election.cc:288] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [CANDIDATE]: Term 1 election: Requested vote from peers 215626758e634a91bd415d06711d8533 (127.8.131.2:41839), 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077)
I20260809 06:25:18.615979 11975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb89440e231a4cd190da263582a33e55" candidate_uuid: "14ee81a1a49a411586a8bd73c4555613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533"
I20260809 06:25:18.616078 12201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb89440e231a4cd190da263582a33e55" candidate_uuid: "14ee81a1a49a411586a8bd73c4555613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4145153bd24ab0beacacd1e288ceea"
I20260809 06:25:18.616199 11975 raft_consensus.cc:3037] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:18.616302 12201 raft_consensus.cc:3037] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:18.618188 11975 raft_consensus.cc:2454] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14ee81a1a49a411586a8bd73c4555613 in term 1.
I20260809 06:25:18.618546 11633 leader_election.cc:302] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14ee81a1a49a411586a8bd73c4555613, 215626758e634a91bd415d06711d8533; no voters:
I20260809 06:25:18.618671 12201 raft_consensus.cc:2454] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14ee81a1a49a411586a8bd73c4555613 in term 1.
I20260809 06:25:18.618855 12327 raft_consensus.cc:2781] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:18.620033 12327 raft_consensus.cc:686] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [term 1 LEADER]: Becoming Leader. State: Replica: 14ee81a1a49a411586a8bd73c4555613, State: Running, Role: LEADER
I20260809 06:25:18.620579 12327 consensus_queue.cc:237] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.623600 12356 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: TxnStatusTablet state changed. Reason: New leader 14ee81a1a49a411586a8bd73c4555613. Latest consensus state: current_term: 1 leader_uuid: "14ee81a1a49a411586a8bd73c4555613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.624063 12356 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: This TxnStatusTablet replica's current role is: LEADER
I20260809 06:25:18.624791 12358 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 06:25:18.625077 12358 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 06:25:18.625519 11471 catalog_manager.cc:5495] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "14ee81a1a49a411586a8bd73c4555613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } health_report { overall_health: UNKNOWN } } }
I20260809 06:25:18.652727 12342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.646617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43385 (local address 127.0.0.1:38516)
0809 06:25:18.646847 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.646869 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.646992 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.647497 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.647500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.647509 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:25:18.647741 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.647751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.649169 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.649172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.650146 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.650157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.651105 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.651109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.651262 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.651321 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:25:18.652314 (+ 993us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:25:18.652334 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.652391 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.652502 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 06:25:18.652869 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.646609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38516 (local address 127.8.131.1:43385)
0809 06:25:18.646815 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.646822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.646947 (+ 125us) server_negotiation.cc:407] Connection header received
0809 06:25:18.647184 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.647190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.647309 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.647483 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:18.647870 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.649055 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.650281 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.650945 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.651101 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.651498 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:18.652189 (+ 691us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:18.652351 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.652444 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.652550 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 06:25:18.660202 11975 raft_consensus.cc:1261] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Refusing update from remote peer 14ee81a1a49a411586a8bd73c4555613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:18.660290 12201 raft_consensus.cc:1261] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Refusing update from remote peer 14ee81a1a49a411586a8bd73c4555613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:18.661160 12356 consensus_queue.cc:1035] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:18.661569 12327 consensus_queue.cc:1035] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:18.675886 12325 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: TxnStatusTablet state changed. Reason: New leader 14ee81a1a49a411586a8bd73c4555613. Latest consensus state: current_term: 1 leader_uuid: "14ee81a1a49a411586a8bd73c4555613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.676195 12325 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:18.685218 12356 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14ee81a1a49a411586a8bd73c4555613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.685491 12356 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: This TxnStatusTablet replica's current role is: LEADER
I20260809 06:25:18.688480 12328 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: TxnStatusTablet state changed. Reason: New leader 14ee81a1a49a411586a8bd73c4555613. Latest consensus state: current_term: 1 leader_uuid: "14ee81a1a49a411586a8bd73c4555613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.688889 12328 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:18.691239 12356 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14ee81a1a49a411586a8bd73c4555613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.691483 12356 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 14ee81a1a49a411586a8bd73c4555613: This TxnStatusTablet replica's current role is: LEADER
I20260809 06:25:18.692871 12328 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14ee81a1a49a411586a8bd73c4555613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.693099 12328 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 215626758e634a91bd415d06711d8533: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:18.693400 12325 tablet_replica.cc:439] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14ee81a1a49a411586a8bd73c4555613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } } }
I20260809 06:25:18.693713 12325 tablet_replica.cc:442] T bb89440e231a4cd190da263582a33e55 P 2a4145153bd24ab0beacacd1e288ceea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:25:18.698179 12365 mvcc.cc:205] Tried to move back new op lower bound from 7316507519622647808 to 7316507519473741824. Current Snapshot: MvccSnapshot[applied={T|T < 7316507519622647808 or (T in {7316507519622647808})}]
I20260809 06:25:18.704615 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.697164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.8.131.3:38077)
0809 06:25:18.697337 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.697344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.699040 (+ 1696us) server_negotiation.cc:407] Connection header received
0809 06:25:18.699290 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.699296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.699397 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.699514 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:18.700085 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.700915 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.702478 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.702892 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.703020 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.703437 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:18.703990 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:18.704144 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.704235 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.704333 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 06:25:18.720590 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.714302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:42205 (local address 127.8.131.1:43385)
0809 06:25:18.714510 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.714517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.715167 (+ 650us) server_negotiation.cc:407] Connection header received
0809 06:25:18.715310 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.715316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.715425 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.715523 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:18.716062 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.717230 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.718503 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.719257 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.719380 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.719811 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:18.719911 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:18.720033 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.720158 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.720239 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 06:25:18.720642 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.714285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43385 (local address 127.8.131.3:42205)
0809 06:25:18.715073 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.715089 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.715196 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.715551 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.715554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.715563 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:25:18.715909 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.715918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.717344 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.717348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.718376 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.718387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.719411 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.719415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.719522 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.719575 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:25:18.720028 (+ 453us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:25:18.720039 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.720136 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.720268 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":437,"threads_started":1}
I20260809 06:25:18.730448 11975 raft_consensus.cc:1261] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Refusing update from remote peer 2a4145153bd24ab0beacacd1e288ceea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:18.730630 11760 raft_consensus.cc:1261] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Refusing update from remote peer 2a4145153bd24ab0beacacd1e288ceea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:18.731185 12332 consensus_queue.cc:1035] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [LEADER]: Connected to new peer: Peer: permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:18.731510 12325 consensus_queue.cc:1035] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [LEADER]: Connected to new peer: Peer: permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:18.736403 12364 mvcc.cc:205] Tried to move back new op lower bound from 7316507519913795584 to 7316507518826741760. Current Snapshot: MvccSnapshot[applied={T|T < 7316507519913795584}]
I20260809 06:25:18.740415 12361 mvcc.cc:205] Tried to move back new op lower bound from 7316507519913795584 to 7316507518826741760. Current Snapshot: MvccSnapshot[applied={T|T < 7316507519913795584}]
I20260809 06:25:18.778570 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.769517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:60301 (local address 127.8.131.3:38077)
0809 06:25:18.769729 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:18.769736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:18.772377 (+ 2641us) server_negotiation.cc:407] Connection header received
0809 06:25:18.772553 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:18.772557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:18.772623 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:18.772704 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:18.773621 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.774646 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.776748 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.777142 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.777238 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.777782 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:18.777891 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:18.777989 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:18.778170 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 06:25:18.778258 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 06:25:18.778721 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:18.768963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:38077 (local address 127.8.131.1:60301)
0809 06:25:18.772264 (+ 3301us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:18.772286 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:18.772433 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:18.772750 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:18.772755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:18.772776 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:25:18.773173 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.773184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.774785 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.774792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.776616 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:18.776633 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:18.777322 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:18.777329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:18.777509 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:18.777583 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:25:18.777993 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:25:18.778007 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 06:25:18.778148 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 06:25:18.778288 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3081,"thread_start_us":2928,"threads_started":1}
I20260809 06:25:18.812381 11471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:18.812646 11471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:18.816143 11471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3: Sending DeleteTablet for 3 replicas of tablet 7c63095f35544854a2d9e325c424399a
I20260809 06:25:18.817263 11719 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c63095f35544854a2d9e325c424399a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:18 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:38512
I20260809 06:25:18.817677 11941 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c63095f35544854a2d9e325c424399a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:18 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:47642
I20260809 06:25:18.818452 12164 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c63095f35544854a2d9e325c424399a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:18 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:37988
I20260809 06:25:18.819934 11471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:25:18.820705 11471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:18.832381 12404 tablet_replica.cc:330] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: stopping tablet replica
I20260809 06:25:18.832882 12404 raft_consensus.cc:2229] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:18.833256 12404 raft_consensus.cc:2258] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:18.834002 12406 tablet_replica.cc:330] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: stopping tablet replica
I20260809 06:25:18.834424 12406 raft_consensus.cc:2229] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:18.834784 12406 raft_consensus.cc:2258] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:18.836140 11941 tablet_service.cc:1467] Processing CreateTablet for tablet 2e1bada4ee11493abd1ec1ae0d33930e (DEFAULT_TABLE table=test-table [id=714d50fbcbf24a0eaf5218492cc62d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:18.836656 11941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e1bada4ee11493abd1ec1ae0d33930e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:18.836705 11718 tablet_service.cc:1467] Processing CreateTablet for tablet 2e1bada4ee11493abd1ec1ae0d33930e (DEFAULT_TABLE table=test-table [id=714d50fbcbf24a0eaf5218492cc62d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:18.837247 11718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e1bada4ee11493abd1ec1ae0d33930e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:18.840921 12163 tablet_service.cc:1467] Processing CreateTablet for tablet 2e1bada4ee11493abd1ec1ae0d33930e (DEFAULT_TABLE table=test-table [id=714d50fbcbf24a0eaf5218492cc62d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:18.841502 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e1bada4ee11493abd1ec1ae0d33930e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:18.844965 12404 ts_tablet_manager.cc:1905] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:18.851629 12406 ts_tablet_manager.cc:1905] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:18.853304 12320 tablet_bootstrap.cc:492] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: Bootstrap starting.
I20260809 06:25:18.855070 12320 tablet_bootstrap.cc:654] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:18.858554 12321 tablet_bootstrap.cc:492] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: Bootstrap starting.
I20260809 06:25:18.860476 12321 tablet_bootstrap.cc:654] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:18.862267 12318 tablet_bootstrap.cc:492] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: Bootstrap starting.
I20260809 06:25:18.864372 12318 tablet_bootstrap.cc:654] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:18.876475 12318 tablet_bootstrap.cc:492] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: No bootstrap required, opened a new log
I20260809 06:25:18.876693 12318 ts_tablet_manager.cc:1397] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.003s
I20260809 06:25:18.877662 12318 raft_consensus.cc:348] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.877965 12318 raft_consensus.cc:374] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:18.878062 12318 raft_consensus.cc:729] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4145153bd24ab0beacacd1e288ceea, State: Initialized, Role: FOLLOWER
I20260809 06:25:18.878286 12320 tablet_bootstrap.cc:492] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: No bootstrap required, opened a new log
I20260809 06:25:18.878404 12318 consensus_queue.cc:260] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.878814 12320 ts_tablet_manager.cc:1397] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.007s
I20260809 06:25:18.879789 12320 raft_consensus.cc:348] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.880118 12320 raft_consensus.cc:374] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:18.881186 12404 ts_tablet_manager.cc:1918] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:25:18.881796 12404 log.cc:1196] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/7c63095f35544854a2d9e325c424399a
I20260809 06:25:18.881939 12318 ts_tablet_manager.cc:1428] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260809 06:25:18.880229 12320 raft_consensus.cc:729] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14ee81a1a49a411586a8bd73c4555613, State: Initialized, Role: FOLLOWER
I20260809 06:25:18.883208 12404 ts_tablet_manager.cc:1939] T 7c63095f35544854a2d9e325c424399a P 14ee81a1a49a411586a8bd73c4555613: Deleting consensus metadata
I20260809 06:25:18.883518 12320 consensus_queue.cc:260] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.885958 11448 catalog_manager.cc:4841] TS 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385): tablet 7c63095f35544854a2d9e325c424399a (table test-table [id=94e931b7f77e4c6d9a291518102ecfba]) successfully deleted
I20260809 06:25:18.887908 12320 ts_tablet_manager.cc:1428] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260809 06:25:18.892154 12407 tablet_replica.cc:330] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: stopping tablet replica
I20260809 06:25:18.892395 12321 tablet_bootstrap.cc:492] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: No bootstrap required, opened a new log
I20260809 06:25:18.892588 12321 ts_tablet_manager.cc:1397] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.002s
I20260809 06:25:18.892648 12407 raft_consensus.cc:2229] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:25:18.893150 12406 ts_tablet_manager.cc:1918] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:25:18.893429 12406 log.cc:1196] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/7c63095f35544854a2d9e325c424399a
I20260809 06:25:18.893698 12407 raft_consensus.cc:2258] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:18.894207 12406 ts_tablet_manager.cc:1939] T 7c63095f35544854a2d9e325c424399a P 215626758e634a91bd415d06711d8533: Deleting consensus metadata
I20260809 06:25:18.893505 12321 raft_consensus.cc:348] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.894886 12321 raft_consensus.cc:374] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:18.894984 12321 raft_consensus.cc:729] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215626758e634a91bd415d06711d8533, State: Initialized, Role: FOLLOWER
I20260809 06:25:18.895251 12321 consensus_queue.cc:260] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:18.895892 12321 ts_tablet_manager.cc:1428] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 06:25:18.896708 11450 catalog_manager.cc:4841] TS 215626758e634a91bd415d06711d8533 (127.8.131.2:41839): tablet 7c63095f35544854a2d9e325c424399a (table test-table [id=94e931b7f77e4c6d9a291518102ecfba]) successfully deleted
I20260809 06:25:18.898878 12407 ts_tablet_manager.cc:1905] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:18.907935 12407 ts_tablet_manager.cc:1918] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:25:18.908133 12407 log.cc:1196] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/7c63095f35544854a2d9e325c424399a
I20260809 06:25:18.908766 12407 ts_tablet_manager.cc:1939] T 7c63095f35544854a2d9e325c424399a P 2a4145153bd24ab0beacacd1e288ceea: Deleting consensus metadata
I20260809 06:25:18.910688 11450 catalog_manager.cc:4841] TS 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077): tablet 7c63095f35544854a2d9e325c424399a (table test-table [id=94e931b7f77e4c6d9a291518102ecfba]) successfully deleted
I20260809 06:25:19.083436 12325 raft_consensus.cc:482] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:19.083698 12325 raft_consensus.cc:504] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:19.084554 12325 leader_election.cc:288] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385), 215626758e634a91bd415d06711d8533 (127.8.131.2:41839)
I20260809 06:25:19.085458 11760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1bada4ee11493abd1ec1ae0d33930e" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14ee81a1a49a411586a8bd73c4555613" is_pre_election: true
I20260809 06:25:19.085736 11760 raft_consensus.cc:2454] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 0.
I20260809 06:25:19.087134 12328 raft_consensus.cc:482] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:19.087407 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1bada4ee11493abd1ec1ae0d33930e" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533" is_pre_election: true
I20260809 06:25:19.087407 12328 raft_consensus.cc:504] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:19.087862 12085 leader_election.cc:302] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14ee81a1a49a411586a8bd73c4555613, 2a4145153bd24ab0beacacd1e288ceea; no voters:
I20260809 06:25:19.088537 12325 raft_consensus.cc:2781] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:19.088696 12325 raft_consensus.cc:482] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:19.088814 12325 raft_consensus.cc:3037] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:19.089661 12328 leader_election.cc:288] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385), 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077)
I20260809 06:25:19.091917 11973 raft_consensus.cc:2454] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 0.
I20260809 06:25:19.092422 12325 raft_consensus.cc:504] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:19.094221 11760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1bada4ee11493abd1ec1ae0d33930e" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14ee81a1a49a411586a8bd73c4555613"
I20260809 06:25:19.094462 11760 raft_consensus.cc:3037] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:19.097833 11760 raft_consensus.cc:2454] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 1.
I20260809 06:25:19.098384 12085 leader_election.cc:302] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14ee81a1a49a411586a8bd73c4555613, 2a4145153bd24ab0beacacd1e288ceea; no voters:
I20260809 06:25:19.098680 12400 raft_consensus.cc:2781] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:19.098883 12400 raft_consensus.cc:686] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 1 LEADER]: Becoming Leader. State: Replica: 2a4145153bd24ab0beacacd1e288ceea, State: Running, Role: LEADER
I20260809 06:25:19.099414 12325 leader_election.cc:288] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 election: Requested vote from peers 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385), 215626758e634a91bd415d06711d8533 (127.8.131.2:41839)
I20260809 06:25:19.099241 12400 consensus_queue.cc:237] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:19.099498 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1bada4ee11493abd1ec1ae0d33930e" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533"
I20260809 06:25:19.099736 11973 raft_consensus.cc:3037] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:19.102973 11973 raft_consensus.cc:2454] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 1.
I20260809 06:25:19.112077 11471 catalog_manager.cc:5495] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3). New cstate: current_term: 1 leader_uuid: "2a4145153bd24ab0beacacd1e288ceea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } health_report { overall_health: HEALTHY } } }
I20260809 06:25:19.117586 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:19.091596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:59765 (local address 127.8.131.1:43385)
0809 06:25:19.091808 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:19.091815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:19.110204 (+ 18389us) server_negotiation.cc:407] Connection header received
0809 06:25:19.110532 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:19.110538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:19.110615 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:19.110703 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:19.111620 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:19.112727 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:19.115099 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:19.116458 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:19.116564 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:19.116664 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:19.117187 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 06:25:19.117315 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 06:25:19.117631 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:19.090280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43385 (local address 127.8.131.2:59765)
0809 06:25:19.110063 (+ 19783us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:19.110081 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:19.110340 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:19.111173 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:19.111180 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:19.111217 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:19.111438 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:19.111450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:19.112940 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:19.112945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:19.114905 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:19.114921 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:19.116815 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:19.116821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:19.117017 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:19.117042 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 06:25:19.117159 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 06:25:19.117293 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19638,"thread_start_us":19534,"threads_started":1}
I20260809 06:25:19.118185 11760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1bada4ee11493abd1ec1ae0d33930e" candidate_uuid: "215626758e634a91bd415d06711d8533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14ee81a1a49a411586a8bd73c4555613" is_pre_election: true
I20260809 06:25:19.118528 11760 raft_consensus.cc:2379] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 215626758e634a91bd415d06711d8533 in current term 1: Already voted for candidate 2a4145153bd24ab0beacacd1e288ceea in this term.
I20260809 06:25:19.127425 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:19.091277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:47037 (local address 127.8.131.3:38077)
0809 06:25:19.091460 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:19.091467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:19.115322 (+ 23855us) server_negotiation.cc:407] Connection header received
0809 06:25:19.115575 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:19.115584 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:19.115711 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:19.115780 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:19.116629 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:19.117975 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:19.123144 (+ 5169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:19.124732 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:19.124850 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:19.124973 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:19.126965 (+ 1992us) server_negotiation.cc:299] Negotiation successful
0809 06:25:19.127124 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:25:19.128225 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:19.090881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:38077 (local address 127.8.131.2:47037)
0809 06:25:19.115204 (+ 24323us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:19.115225 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:19.115427 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:19.115976 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:19.115981 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:19.116010 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:19.116447 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:19.116459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:19.120827 (+ 4368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:19.120833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:19.122867 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:19.122939 (+ 72us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:19.126500 (+ 3561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:19.126512 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:19.126738 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:19.126776 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 06:25:19.127711 (+ 935us) client_negotiation.cc:240] Negotiation successful
0809 06:25:19.127863 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24203,"thread_start_us":24019,"threads_started":1}
I20260809 06:25:19.128966 12201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1bada4ee11493abd1ec1ae0d33930e" candidate_uuid: "215626758e634a91bd415d06711d8533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4145153bd24ab0beacacd1e288ceea" is_pre_election: true
I20260809 06:25:19.129736 11862 leader_election.cc:302] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 215626758e634a91bd415d06711d8533; no voters: 14ee81a1a49a411586a8bd73c4555613, 2a4145153bd24ab0beacacd1e288ceea
I20260809 06:25:19.130282 12328 raft_consensus.cc:2726] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 06:25:19.186594 11760 raft_consensus.cc:1261] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Refusing update from remote peer 2a4145153bd24ab0beacacd1e288ceea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:19.186614 11973 raft_consensus.cc:1261] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Refusing update from remote peer 2a4145153bd24ab0beacacd1e288ceea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:19.187264 12325 consensus_queue.cc:1035] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [LEADER]: Connected to new peer: Peer: permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:19.187579 12400 consensus_queue.cc:1035] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [LEADER]: Connected to new peer: Peer: permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 06:25:19.215176 12364 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 06:25:20.804154 11472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:20.804402 11472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:20.808377 11472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3: Sending DeleteTablet for 3 replicas of tablet 2e1bada4ee11493abd1ec1ae0d33930e
I20260809 06:25:20.809185 11718 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e1bada4ee11493abd1ec1ae0d33930e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:20 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:38512
I20260809 06:25:20.809324 11941 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e1bada4ee11493abd1ec1ae0d33930e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:20 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:47642
I20260809 06:25:20.809403 12164 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e1bada4ee11493abd1ec1ae0d33930e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:20 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:37988
I20260809 06:25:20.814881 12462 tablet_replica.cc:330] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: stopping tablet replica
I20260809 06:25:20.815181 12462 raft_consensus.cc:2229] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:20.815493 12462 raft_consensus.cc:2258] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:20.816088 12463 tablet_replica.cc:330] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: stopping tablet replica
I20260809 06:25:20.816355 12463 raft_consensus.cc:2229] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:25:20.816679 12462 ts_tablet_manager.cc:1905] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:20.816754 12463 raft_consensus.cc:2258] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:20.817682 12463 ts_tablet_manager.cc:1905] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:20.821698 12459 tablet_replica.cc:330] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: stopping tablet replica
I20260809 06:25:20.821985 12459 raft_consensus.cc:2229] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:20.822286 12459 raft_consensus.cc:2258] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:20.823495 12459 ts_tablet_manager.cc:1905] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:20.823969 12462 ts_tablet_manager.cc:1918] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 06:25:20.824239 12462 log.cc:1196] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/2e1bada4ee11493abd1ec1ae0d33930e
I20260809 06:25:20.824260 12463 ts_tablet_manager.cc:1918] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 06:25:20.824416 12463 log.cc:1196] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/2e1bada4ee11493abd1ec1ae0d33930e
I20260809 06:25:20.824877 12463 ts_tablet_manager.cc:1939] T 2e1bada4ee11493abd1ec1ae0d33930e P 2a4145153bd24ab0beacacd1e288ceea: Deleting consensus metadata
I20260809 06:25:20.824875 12462 ts_tablet_manager.cc:1939] T 2e1bada4ee11493abd1ec1ae0d33930e P 215626758e634a91bd415d06711d8533: Deleting consensus metadata
I20260809 06:25:20.826054 11450 catalog_manager.cc:4841] TS 215626758e634a91bd415d06711d8533 (127.8.131.2:41839): tablet 2e1bada4ee11493abd1ec1ae0d33930e (table test-table [id=714d50fbcbf24a0eaf5218492cc62d53]) successfully deleted
I20260809 06:25:20.826327 11450 catalog_manager.cc:4841] TS 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077): tablet 2e1bada4ee11493abd1ec1ae0d33930e (table test-table [id=714d50fbcbf24a0eaf5218492cc62d53]) successfully deleted
I20260809 06:25:20.831214 12459 ts_tablet_manager.cc:1918] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 06:25:20.831364 12459 log.cc:1196] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/2e1bada4ee11493abd1ec1ae0d33930e
I20260809 06:25:20.831898 12459 ts_tablet_manager.cc:1939] T 2e1bada4ee11493abd1ec1ae0d33930e P 14ee81a1a49a411586a8bd73c4555613: Deleting consensus metadata
I20260809 06:25:20.833156 11448 catalog_manager.cc:4841] TS 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385): tablet 2e1bada4ee11493abd1ec1ae0d33930e (table test-table [id=714d50fbcbf24a0eaf5218492cc62d53]) successfully deleted
Aug 09 06:25:20 dist-test-slave-mjjr krb5kdc[11399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 06:25:20 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:20.869632 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:20.854491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.8.131.62:37599)
0809 06:25:20.855509 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:20.855514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:20.855536 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:25:20.855593 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:20.855596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:20.855731 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:20.855826 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:20.857284 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:20.858077 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:20.859317 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:20.859816 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:20.859925 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:20.863788 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:20.863794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:20.863797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:20.863828 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:20.866418 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:20.867232 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:20.867241 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:20.867246 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:20.867408 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:20.867908 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:20.867914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:20.867919 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:20.868222 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:20.868500 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:20.868890 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 06:25:20.869133 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":901,"thread_start_us":701,"threads_started":1}
I20260809 06:25:20.873128 11472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:25:20.873736 11472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:20.879906 11718 tablet_service.cc:1467] Processing CreateTablet for tablet 685f8acf18f24ad196ca63f6254f2fbf (DEFAULT_TABLE table=test-table [id=52c4b198a181457bb586f3b89cc507f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:20.880453 11718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685f8acf18f24ad196ca63f6254f2fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:20.880352 11941 tablet_service.cc:1467] Processing CreateTablet for tablet 685f8acf18f24ad196ca63f6254f2fbf (DEFAULT_TABLE table=test-table [id=52c4b198a181457bb586f3b89cc507f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:20.880466 12164 tablet_service.cc:1467] Processing CreateTablet for tablet 685f8acf18f24ad196ca63f6254f2fbf (DEFAULT_TABLE table=test-table [id=52c4b198a181457bb586f3b89cc507f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:20.880797 11941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685f8acf18f24ad196ca63f6254f2fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:20.880872 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685f8acf18f24ad196ca63f6254f2fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:20.887876 12478 tablet_bootstrap.cc:492] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: Bootstrap starting.
I20260809 06:25:20.888706 12476 tablet_bootstrap.cc:492] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: Bootstrap starting.
I20260809 06:25:20.889904 12478 tablet_bootstrap.cc:654] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:20.891197 12481 tablet_bootstrap.cc:492] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: Bootstrap starting.
I20260809 06:25:20.891309 12476 tablet_bootstrap.cc:654] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:20.892647 12481 tablet_bootstrap.cc:654] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:20.893213 12476 tablet_bootstrap.cc:492] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: No bootstrap required, opened a new log
I20260809 06:25:20.893376 12476 ts_tablet_manager.cc:1397] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:25:20.894356 12476 raft_consensus.cc:348] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:20.894549 12478 tablet_bootstrap.cc:492] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: No bootstrap required, opened a new log
I20260809 06:25:20.894572 12476 raft_consensus.cc:374] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:20.894670 12476 raft_consensus.cc:729] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215626758e634a91bd415d06711d8533, State: Initialized, Role: FOLLOWER
I20260809 06:25:20.894706 12478 ts_tablet_manager.cc:1397] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260809 06:25:20.894923 12476 consensus_queue.cc:260] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:20.895444 12478 raft_consensus.cc:348] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:20.895701 12476 ts_tablet_manager.cc:1428] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 06:25:20.895738 12478 raft_consensus.cc:374] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:20.895876 12478 raft_consensus.cc:729] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4145153bd24ab0beacacd1e288ceea, State: Initialized, Role: FOLLOWER
I20260809 06:25:20.896159 12478 consensus_queue.cc:260] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:20.896744 12478 ts_tablet_manager.cc:1428] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 06:25:20.898013 12481 tablet_bootstrap.cc:492] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: No bootstrap required, opened a new log
I20260809 06:25:20.898175 12481 ts_tablet_manager.cc:1397] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260809 06:25:20.899185 12481 raft_consensus.cc:348] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:20.899454 12481 raft_consensus.cc:374] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:20.899561 12481 raft_consensus.cc:729] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14ee81a1a49a411586a8bd73c4555613, State: Initialized, Role: FOLLOWER
I20260809 06:25:20.899857 12481 consensus_queue.cc:260] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:20.900468 12481 ts_tablet_manager.cc:1428] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 06:25:20.979174 12424 raft_consensus.cc:482] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:20.979410 12424 raft_consensus.cc:504] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:20.980275 12424 leader_election.cc:288] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 215626758e634a91bd415d06711d8533 (127.8.131.2:41839), 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077)
I20260809 06:25:20.980796 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685f8acf18f24ad196ca63f6254f2fbf" candidate_uuid: "14ee81a1a49a411586a8bd73c4555613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533" is_pre_election: true
I20260809 06:25:20.980842 12201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685f8acf18f24ad196ca63f6254f2fbf" candidate_uuid: "14ee81a1a49a411586a8bd73c4555613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4145153bd24ab0beacacd1e288ceea" is_pre_election: true
I20260809 06:25:20.981050 11973 raft_consensus.cc:2454] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14ee81a1a49a411586a8bd73c4555613 in term 0.
I20260809 06:25:20.981112 12201 raft_consensus.cc:2454] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14ee81a1a49a411586a8bd73c4555613 in term 0.
I20260809 06:25:20.981577 11633 leader_election.cc:302] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14ee81a1a49a411586a8bd73c4555613, 215626758e634a91bd415d06711d8533; no voters:
I20260809 06:25:20.981875 12424 raft_consensus.cc:2781] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:20.982002 12424 raft_consensus.cc:482] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:20.982095 12424 raft_consensus.cc:3037] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:20.984529 12424 raft_consensus.cc:504] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:20.985059 12424 leader_election.cc:288] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [CANDIDATE]: Term 1 election: Requested vote from peers 215626758e634a91bd415d06711d8533 (127.8.131.2:41839), 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077)
I20260809 06:25:20.985594 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685f8acf18f24ad196ca63f6254f2fbf" candidate_uuid: "14ee81a1a49a411586a8bd73c4555613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533"
I20260809 06:25:20.985634 12201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685f8acf18f24ad196ca63f6254f2fbf" candidate_uuid: "14ee81a1a49a411586a8bd73c4555613" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4145153bd24ab0beacacd1e288ceea"
I20260809 06:25:20.985819 11973 raft_consensus.cc:3037] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:20.985855 12201 raft_consensus.cc:3037] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:20.988106 12201 raft_consensus.cc:2454] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14ee81a1a49a411586a8bd73c4555613 in term 1.
I20260809 06:25:20.988343 11973 raft_consensus.cc:2454] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14ee81a1a49a411586a8bd73c4555613 in term 1.
I20260809 06:25:20.988528 11633 leader_election.cc:302] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14ee81a1a49a411586a8bd73c4555613, 2a4145153bd24ab0beacacd1e288ceea; no voters:
I20260809 06:25:20.988839 12424 raft_consensus.cc:2781] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:20.988998 12424 raft_consensus.cc:686] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 1 LEADER]: Becoming Leader. State: Replica: 14ee81a1a49a411586a8bd73c4555613, State: Running, Role: LEADER
I20260809 06:25:20.989297 12424 consensus_queue.cc:237] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:20.992054 11471 catalog_manager.cc:5495] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "14ee81a1a49a411586a8bd73c4555613" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } health_report { overall_health: UNKNOWN } } }
I20260809 06:25:21.024816 12493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:21.015437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38532 (local address 127.8.131.1:43385)
0809 06:25:21.020087 (+ 4650us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:21.020092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:21.020110 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:25:21.020158 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:21.020162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:21.020218 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:21.020353 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:21.020772 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:21.021543 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:21.022923 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:21.023345 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:21.023507 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:21.024037 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:21.024154 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:21.024256 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:21.024406 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 06:25:21.024484 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4550,"thread_start_us":4398,"threads_started":1}
I20260809 06:25:21.037911 12493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:21.032970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:34341 (local address 127.8.131.1:43385)
0809 06:25:21.033111 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:21.033115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:21.033614 (+ 499us) server_negotiation.cc:407] Connection header received
0809 06:25:21.033745 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:21.033748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:21.033805 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:21.033878 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:21.034281 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:21.035017 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:21.036362 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:21.036734 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:21.036902 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:21.037274 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:21.037386 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:21.037454 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:21.037574 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 06:25:21.037652 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":49}
I20260809 06:25:21.037998 12500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:21.032965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43385 (local address 127.8.131.1:34341)
0809 06:25:21.033522 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:21.033537 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:21.033647 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:21.033905 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:21.033908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:21.033916 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:25:21.034167 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:21.034174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:21.035123 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:21.035127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:21.036243 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:21.036255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:21.036909 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:21.036913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:21.037104 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:21.037139 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:25:21.037473 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:25:21.037481 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 06:25:21.037540 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 06:25:21.037648 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":2,"thread_start_us":320,"threads_started":1}
I20260809 06:25:21.045548 12201 raft_consensus.cc:1261] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Refusing update from remote peer 14ee81a1a49a411586a8bd73c4555613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:21.045594 11973 raft_consensus.cc:1261] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Refusing update from remote peer 14ee81a1a49a411586a8bd73c4555613: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:21.046151 12391 consensus_queue.cc:1035] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:21.046563 12452 consensus_queue.cc:1035] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:21.102375 11471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:21.102639 11471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:21.106227 11471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3: Sending DeleteTablet for 3 replicas of tablet 685f8acf18f24ad196ca63f6254f2fbf
I20260809 06:25:21.107347 11718 tablet_service.cc:1514] Processing DeleteTablet for tablet 685f8acf18f24ad196ca63f6254f2fbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:21 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:38512
I20260809 06:25:21.107831 12459 tablet_replica.cc:330] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: stopping tablet replica
I20260809 06:25:21.107775 11941 tablet_service.cc:1514] Processing DeleteTablet for tablet 685f8acf18f24ad196ca63f6254f2fbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:21 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:47642
I20260809 06:25:21.108083 12459 raft_consensus.cc:2229] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:25:21.108114 12462 tablet_replica.cc:330] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: stopping tablet replica
I20260809 06:25:21.108199 12164 tablet_service.cc:1514] Processing DeleteTablet for tablet 685f8acf18f24ad196ca63f6254f2fbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:21 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:37988
I20260809 06:25:21.108448 12462 raft_consensus.cc:2229] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:21.108597 12459 raft_consensus.cc:2258] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:21.108800 12462 raft_consensus.cc:2258] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:21.109229 12463 tablet_replica.cc:330] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: stopping tablet replica
I20260809 06:25:21.109459 12463 raft_consensus.cc:2229] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:21.109694 12463 raft_consensus.cc:2258] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:21.110025 12459 ts_tablet_manager.cc:1905] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:21.109951 11471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:25:21.110620 11471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:21.114153 12463 ts_tablet_manager.cc:1905] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:21.117317 12459 ts_tablet_manager.cc:1918] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:25:21.117560 12459 log.cc:1196] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/685f8acf18f24ad196ca63f6254f2fbf
I20260809 06:25:21.118046 12459 ts_tablet_manager.cc:1939] T 685f8acf18f24ad196ca63f6254f2fbf P 14ee81a1a49a411586a8bd73c4555613: Deleting consensus metadata
I20260809 06:25:21.118719 12164 tablet_service.cc:1467] Processing CreateTablet for tablet 9cad9b60db4f4edba2fae89a26d66c81 (DEFAULT_TABLE table=test-table [id=fd931ea0f0614dfdac7a4f4f3691aac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:21.119184 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cad9b60db4f4edba2fae89a26d66c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:21.121397 12463 ts_tablet_manager.cc:1918] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:25:21.121554 12463 log.cc:1196] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/685f8acf18f24ad196ca63f6254f2fbf
I20260809 06:25:21.121992 12463 ts_tablet_manager.cc:1939] T 685f8acf18f24ad196ca63f6254f2fbf P 2a4145153bd24ab0beacacd1e288ceea: Deleting consensus metadata
I20260809 06:25:21.122818 11941 tablet_service.cc:1467] Processing CreateTablet for tablet 9cad9b60db4f4edba2fae89a26d66c81 (DEFAULT_TABLE table=test-table [id=fd931ea0f0614dfdac7a4f4f3691aac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:21.123097 11450 catalog_manager.cc:4841] TS 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077): tablet 685f8acf18f24ad196ca63f6254f2fbf (table test-table [id=52c4b198a181457bb586f3b89cc507f9]) successfully deleted
I20260809 06:25:21.123301 11941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cad9b60db4f4edba2fae89a26d66c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:21.124076 11448 catalog_manager.cc:4841] TS 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385): tablet 685f8acf18f24ad196ca63f6254f2fbf (table test-table [id=52c4b198a181457bb586f3b89cc507f9]) successfully deleted
I20260809 06:25:21.128104 11718 tablet_service.cc:1467] Processing CreateTablet for tablet 9cad9b60db4f4edba2fae89a26d66c81 (DEFAULT_TABLE table=test-table [id=fd931ea0f0614dfdac7a4f4f3691aac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:21.128574 11718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cad9b60db4f4edba2fae89a26d66c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:21.128671 12478 tablet_bootstrap.cc:492] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: Bootstrap starting.
I20260809 06:25:21.130452 12478 tablet_bootstrap.cc:654] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:21.138044 12462 ts_tablet_manager.cc:1905] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:21.140223 12476 tablet_bootstrap.cc:492] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: Bootstrap starting.
I20260809 06:25:21.142414 12476 tablet_bootstrap.cc:654] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:21.145309 12462 ts_tablet_manager.cc:1918] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:25:21.145568 12462 log.cc:1196] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/685f8acf18f24ad196ca63f6254f2fbf
I20260809 06:25:21.146097 12462 ts_tablet_manager.cc:1939] T 685f8acf18f24ad196ca63f6254f2fbf P 215626758e634a91bd415d06711d8533: Deleting consensus metadata
I20260809 06:25:21.147114 12481 tablet_bootstrap.cc:492] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: Bootstrap starting.
I20260809 06:25:21.149466 12481 tablet_bootstrap.cc:654] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:21.150748 11450 catalog_manager.cc:4841] TS 215626758e634a91bd415d06711d8533 (127.8.131.2:41839): tablet 685f8acf18f24ad196ca63f6254f2fbf (table test-table [id=52c4b198a181457bb586f3b89cc507f9]) successfully deleted
I20260809 06:25:21.152529 12476 tablet_bootstrap.cc:492] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: No bootstrap required, opened a new log
I20260809 06:25:21.152709 12476 ts_tablet_manager.cc:1397] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.009s
I20260809 06:25:21.153509 12476 raft_consensus.cc:348] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:21.153780 12476 raft_consensus.cc:374] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:21.153885 12476 raft_consensus.cc:729] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215626758e634a91bd415d06711d8533, State: Initialized, Role: FOLLOWER
I20260809 06:25:21.154242 12476 consensus_queue.cc:260] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:21.154838 12476 ts_tablet_manager.cc:1428] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 06:25:21.156420 12478 tablet_bootstrap.cc:492] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: No bootstrap required, opened a new log
I20260809 06:25:21.156659 12478 ts_tablet_manager.cc:1397] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.014s
I20260809 06:25:21.157546 12478 raft_consensus.cc:348] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:21.157917 12478 raft_consensus.cc:374] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:21.158030 12478 raft_consensus.cc:729] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4145153bd24ab0beacacd1e288ceea, State: Initialized, Role: FOLLOWER
I20260809 06:25:21.158298 12478 consensus_queue.cc:260] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:21.158912 12478 ts_tablet_manager.cc:1428] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 06:25:21.163116 12481 tablet_bootstrap.cc:492] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: No bootstrap required, opened a new log
I20260809 06:25:21.163345 12481 ts_tablet_manager.cc:1397] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.003s
I20260809 06:25:21.164122 12481 raft_consensus.cc:348] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:21.164361 12481 raft_consensus.cc:374] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:21.164479 12481 raft_consensus.cc:729] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14ee81a1a49a411586a8bd73c4555613, State: Initialized, Role: FOLLOWER
I20260809 06:25:21.164741 12481 consensus_queue.cc:260] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:21.165722 12481 ts_tablet_manager.cc:1428] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 06:25:21.274888 12445 raft_consensus.cc:482] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:21.275141 12445 raft_consensus.cc:504] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:21.275955 12445 leader_election.cc:288] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385), 215626758e634a91bd415d06711d8533 (127.8.131.2:41839)
I20260809 06:25:21.276561 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cad9b60db4f4edba2fae89a26d66c81" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533" is_pre_election: true
I20260809 06:25:21.276568 11759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cad9b60db4f4edba2fae89a26d66c81" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14ee81a1a49a411586a8bd73c4555613" is_pre_election: true
I20260809 06:25:21.276815 11973 raft_consensus.cc:2454] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 0.
I20260809 06:25:21.276861 11759 raft_consensus.cc:2454] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 0.
I20260809 06:25:21.277367 12085 leader_election.cc:302] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14ee81a1a49a411586a8bd73c4555613, 2a4145153bd24ab0beacacd1e288ceea; no voters:
I20260809 06:25:21.277647 12445 raft_consensus.cc:2781] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:21.277814 12445 raft_consensus.cc:482] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:21.277909 12445 raft_consensus.cc:3037] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:21.280265 12445 raft_consensus.cc:504] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:21.280825 12445 leader_election.cc:288] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 election: Requested vote from peers 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385), 215626758e634a91bd415d06711d8533 (127.8.131.2:41839)
I20260809 06:25:21.281333 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cad9b60db4f4edba2fae89a26d66c81" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215626758e634a91bd415d06711d8533"
I20260809 06:25:21.281539 11973 raft_consensus.cc:3037] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:21.283934 11759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cad9b60db4f4edba2fae89a26d66c81" candidate_uuid: "2a4145153bd24ab0beacacd1e288ceea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14ee81a1a49a411586a8bd73c4555613"
I20260809 06:25:21.284014 11973 raft_consensus.cc:2454] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 1.
I20260809 06:25:21.284191 11759 raft_consensus.cc:3037] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:21.284493 12086 leader_election.cc:302] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215626758e634a91bd415d06711d8533, 2a4145153bd24ab0beacacd1e288ceea; no voters:
I20260809 06:25:21.284765 12445 raft_consensus.cc:2781] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:21.284961 12445 raft_consensus.cc:686] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 1 LEADER]: Becoming Leader. State: Replica: 2a4145153bd24ab0beacacd1e288ceea, State: Running, Role: LEADER
I20260809 06:25:21.285274 12445 consensus_queue.cc:237] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } }
I20260809 06:25:21.287406 11759 raft_consensus.cc:2454] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a4145153bd24ab0beacacd1e288ceea in term 1.
I20260809 06:25:21.289371 11471 catalog_manager.cc:5495] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3). New cstate: current_term: 1 leader_uuid: "2a4145153bd24ab0beacacd1e288ceea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a4145153bd24ab0beacacd1e288ceea" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 38077 } health_report { overall_health: HEALTHY } } }
I20260809 06:25:21.347750 12513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:21.341715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.8.131.3:38077)
0809 06:25:21.342234 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:21.342238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:21.342260 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:25:21.342393 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:21.342397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:21.342465 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:21.342544 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:21.342982 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:21.343854 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:21.345841 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:21.346243 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:21.346338 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:21.346889 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:21.346996 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:21.347064 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:21.347247 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 06:25:21.347307 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":298,"threads_started":1}
I20260809 06:25:21.357400 11973 raft_consensus.cc:1261] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Refusing update from remote peer 2a4145153bd24ab0beacacd1e288ceea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:21.357599 11759 raft_consensus.cc:1261] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Refusing update from remote peer 2a4145153bd24ab0beacacd1e288ceea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:21.358050 12445 consensus_queue.cc:1035] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [LEADER]: Connected to new peer: Peer: permanent_uuid: "215626758e634a91bd415d06711d8533" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:21.358351 12440 consensus_queue.cc:1035] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [LEADER]: Connected to new peer: Peer: permanent_uuid: "14ee81a1a49a411586a8bd73c4555613" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 06:25:21.383224 12504 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 06:25:22.978859 11471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:22.979164 11471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:22.983496 11471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c8db8228820422e984f6d6645b0e4b3: Sending DeleteTablet for 3 replicas of tablet 9cad9b60db4f4edba2fae89a26d66c81
I20260809 06:25:22.984395 11719 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cad9b60db4f4edba2fae89a26d66c81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:22 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:38512
I20260809 06:25:22.984452 12164 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cad9b60db4f4edba2fae89a26d66c81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:22 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:37988
I20260809 06:25:22.984534 11941 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cad9b60db4f4edba2fae89a26d66c81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:22 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:47642
I20260809 06:25:22.985271 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 11606
I20260809 06:25:22.989391 12548 tablet_replica.cc:330] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: stopping tablet replica
I20260809 06:25:22.989693 12548 raft_consensus.cc:2229] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:22.990123 12548 raft_consensus.cc:2258] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:22.990193 12549 tablet_replica.cc:330] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: stopping tablet replica
I20260809 06:25:22.990387 12549 raft_consensus.cc:2229] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:25:22.990805 12549 raft_consensus.cc:2258] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:22.991426 12550 tablet_replica.cc:330] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: stopping tablet replica
I20260809 06:25:22.991592 12548 ts_tablet_manager.cc:1905] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:22.991717 12550 raft_consensus.cc:2229] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:22.991987 12550 raft_consensus.cc:2258] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:22.992799 12549 ts_tablet_manager.cc:1905] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:22.992975 12550 ts_tablet_manager.cc:1905] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:22.999085 12548 ts_tablet_manager.cc:1918] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 06:25:22.999310 12548 log.cc:1196] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/9cad9b60db4f4edba2fae89a26d66c81
I20260809 06:25:22.999956 12550 ts_tablet_manager.cc:1918] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 06:25:22.999994 12548 ts_tablet_manager.cc:1939] T 9cad9b60db4f4edba2fae89a26d66c81 P 14ee81a1a49a411586a8bd73c4555613: Deleting consensus metadata
I20260809 06:25:23.000180 12550 log.cc:1196] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/9cad9b60db4f4edba2fae89a26d66c81
I20260809 06:25:23.000237 12549 ts_tablet_manager.cc:1918] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 06:25:23.000447 12549 log.cc:1196] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/9cad9b60db4f4edba2fae89a26d66c81
I20260809 06:25:23.000795 12550 ts_tablet_manager.cc:1939] T 9cad9b60db4f4edba2fae89a26d66c81 P 215626758e634a91bd415d06711d8533: Deleting consensus metadata
I20260809 06:25:23.001032 12549 ts_tablet_manager.cc:1939] T 9cad9b60db4f4edba2fae89a26d66c81 P 2a4145153bd24ab0beacacd1e288ceea: Deleting consensus metadata
I20260809 06:25:23.001444 11448 catalog_manager.cc:4841] TS 14ee81a1a49a411586a8bd73c4555613 (127.8.131.1:43385): tablet 9cad9b60db4f4edba2fae89a26d66c81 (table test-table [id=fd931ea0f0614dfdac7a4f4f3691aac3]) successfully deleted
Aug 09 06:25:22 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:23.002207 11450 catalog_manager.cc:4841] TS 215626758e634a91bd415d06711d8533 (127.8.131.2:41839): tablet 9cad9b60db4f4edba2fae89a26d66c81 (table test-table [id=fd931ea0f0614dfdac7a4f4f3691aac3]) successfully deleted
I20260809 06:25:23.002454 11450 catalog_manager.cc:4841] TS 2a4145153bd24ab0beacacd1e288ceea (127.8.131.3:38077): tablet 9cad9b60db4f4edba2fae89a26d66c81 (table test-table [id=fd931ea0f0614dfdac7a4f4f3691aac3]) successfully deleted
I20260809 06:25:23.008842 12557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:22.987597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38538 (local address 127.8.131.1:43385)
0809 06:25:22.991590 (+ 3993us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:22.991599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:22.991625 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:25:22.991713 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:22.991718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:22.991792 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:22.991978 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:22.993726 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:22.994864 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:22.996827 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:22.997595 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:22.998350 (+ 755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:23.003079 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:23.003105 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:23.003122 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:23.003159 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:23.005927 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:23.006649 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:23.006659 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:23.006662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:23.006810 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:23.007199 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:23.007204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:23.007207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:23.007572 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:23.007932 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:23.008222 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 06:25:23.008465 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3845,"thread_start_us":4042,"threads_started":1}
W20260809 06:25:23.009546 11828 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38538
W20260809 06:25:23.010200 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 11606: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:25:23.010445 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 11606
I20260809 06:25:23.048216 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 11840
Aug 09 06:25:23 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:25:23.063292 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:23.049039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47658 (local address 127.8.131.2:41839)
0809 06:25:23.049527 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:23.049532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:23.049549 (+ 17us) server_negotiation.cc:407] Connection header received
0809 06:25:23.049606 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:23.049609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:23.049671 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:23.049761 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:23.050964 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:23.051834 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:23.053037 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:23.053420 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:23.053537 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:23.057765 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:23.057780 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:23.057796 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:23.057835 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:23.060843 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:23.061479 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:23.061486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:23.061488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:23.061619 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:23.061931 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:23.061934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:23.061936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:23.062263 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:23.062532 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:23.062859 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 06:25:23.063028 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":245,"threads_started":1}
W20260809 06:25:23.064044 12044 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47658
W20260809 06:25:23.064708 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 11840: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:25:23.064872 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 11840
I20260809 06:25:23.087721 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 12055
Aug 09 06:25:23 dist-test-slave-mjjr krb5kdc[11399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:25:23.104715 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:23.088584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.8.131.3:38077)
0809 06:25:23.089234 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:23.089239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:23.089255 (+ 16us) server_negotiation.cc:407] Connection header received
0809 06:25:23.089311 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:23.089315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:23.089390 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:23.089551 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:23.090885 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:23.091617 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:23.092894 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:23.093302 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:23.093443 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:23.098504 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:23.098521 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:23.098540 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:23.098581 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:23.101869 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:23.102532 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:23.102540 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:23.102542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:23.102690 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:23.103075 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:23.103080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:23.103084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:23.103458 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:23.103825 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:23.104112 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 06:25:23.104358 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":279,"threads_started":1}
W20260809 06:25:23.105392 12275 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38020
W20260809 06:25:23.106070 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 12055: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:25:23.106243 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 12055
I20260809 06:25:23.133186 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 11421
I20260809 06:25:23.134425 11547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:23.252445 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 11421
2026-08-09T06:25:23Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8753 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:23 dist-test-slave-mjjr krb5kdc[12565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:23 dist-test-slave-mjjr krb5kdc[12565](info): set up 2 sockets
Aug 09 06:25:23 dist-test-slave-mjjr krb5kdc[12565](info): commencing operation
krb5kdc: starting...
W20260809 06:25:24.568958 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:24 dist-test-slave-mjjr krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:24.777140 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:40407
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:40407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:25.102895 12586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:25.103286 12586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:25.103466 12586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:25.110623 12586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:25.110772 12586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:25.110877 12586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:25.110967 12586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:25.111034 12586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:25.120705 12586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:40407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:40407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:25.121989 12586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:25.123281 12586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:25.135937 12594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:25.135903 12595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:25.135988 12600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:25.136942 12586 server_base.cc:1034] running on GCE node
I20260809 06:25:25.137712 12586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:25.139204 12586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:25.140538 12586 hybrid_clock.cc:648] HybridClock initialized: now 1786256725140414 us; error 97 us; skew 500 ppm
I20260809 06:25:25.141008 12586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:25 dist-test-slave-mjjr krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256725, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:25.145516 12586 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:25.147308 12586 webserver.cc:460] Webserver started at http://127.8.131.62:46037/ using document root <none> and password file <none>
I20260809 06:25:25.148296 12586 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:25.148425 12586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:25.148692 12586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:25.151695 12586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "16c8e5dd75a8410e8605babf728dde34"
format_stamp: "Formatted at 2026-08-09 06:25:25 on dist-test-slave-mjjr"
server_key: "026998f17d365595c59c7bb739586cb4"
server_key_iv: "0c915aebf0eac1457c952d5c871e0f76"
server_key_version: "encryptionkey@0"
I20260809 06:25:25.152464 12586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "16c8e5dd75a8410e8605babf728dde34"
format_stamp: "Formatted at 2026-08-09 06:25:25 on dist-test-slave-mjjr"
server_key: "026998f17d365595c59c7bb739586cb4"
server_key_iv: "0c915aebf0eac1457c952d5c871e0f76"
server_key_version: "encryptionkey@0"
I20260809 06:25:25.157603 12586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:25.160884 12608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:25.162204 12586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:25.162395 12586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "16c8e5dd75a8410e8605babf728dde34"
format_stamp: "Formatted at 2026-08-09 06:25:25 on dist-test-slave-mjjr"
server_key: "026998f17d365595c59c7bb739586cb4"
server_key_iv: "0c915aebf0eac1457c952d5c871e0f76"
server_key_version: "encryptionkey@0"
I20260809 06:25:25.162586 12586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:25.190027 12586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:25.193974 12586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:25.194262 12586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:25.213263 12586 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:40407
I20260809 06:25:25.213296 12681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:40407 every 8 connection(s)
I20260809 06:25:25.214704 12586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:25.216787 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 12586
I20260809 06:25:25.217029 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:25.217849 8716 external_mini_cluster.cc:1428] Setting key 2843b2db571c7fbfefb6519d1372469e
I20260809 06:25:25.219251 12682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 06:25:25 dist-test-slave-mjjr krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:25.233412 12682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34: Bootstrap starting.
I20260809 06:25:25.236631 12682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:25.237984 12682 log.cc:824] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:25.241401 12682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34: No bootstrap required, opened a new log
I20260809 06:25:25.241619 12686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:25.219979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.8.131.62:40407)
0809 06:25:25.220763 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:25.220774 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:25.220816 (+ 42us) server_negotiation.cc:407] Connection header received
0809 06:25:25.222383 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:25.222402 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:25.222778 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:25.223105 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:25.224916 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:25.226236 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:25.228071 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:25.228725 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:25.228852 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:25.233500 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:25.233508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:25.233522 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:25.233589 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:25.236709 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:25.237595 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:25.237601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:25.237607 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:25.237788 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:25.238247 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:25.238252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:25.238256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:25.239078 (+ 822us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:25.239397 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:25.239845 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 06:25:25.240274 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":365,"threads_started":1}
I20260809 06:25:25.249298 12682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c8e5dd75a8410e8605babf728dde34" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40407 } }
I20260809 06:25:25.249641 12682 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:25.249714 12682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c8e5dd75a8410e8605babf728dde34, State: Initialized, Role: FOLLOWER
I20260809 06:25:25.250304 12682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c8e5dd75a8410e8605babf728dde34" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40407 } }
I20260809 06:25:25.250591 12682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:25.250751 12682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:25.250947 12682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:25.253124 12682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c8e5dd75a8410e8605babf728dde34" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40407 } }
I20260809 06:25:25.253594 12682 leader_election.cc:302] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16c8e5dd75a8410e8605babf728dde34; no voters:
I20260809 06:25:25.254143 12682 leader_election.cc:288] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:25.254293 12691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:25.254978 12691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [term 1 LEADER]: Becoming Leader. State: Replica: 16c8e5dd75a8410e8605babf728dde34, State: Running, Role: LEADER
I20260809 06:25:25.255548 12691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c8e5dd75a8410e8605babf728dde34" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40407 } }
I20260809 06:25:25.256233 12682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:25.258706 12694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16c8e5dd75a8410e8605babf728dde34. Latest consensus state: current_term: 1 leader_uuid: "16c8e5dd75a8410e8605babf728dde34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c8e5dd75a8410e8605babf728dde34" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40407 } } }
I20260809 06:25:25.258744 12693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16c8e5dd75a8410e8605babf728dde34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c8e5dd75a8410e8605babf728dde34" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40407 } } }
I20260809 06:25:25.259169 12694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:25.259166 12693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:25.262707 12706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:25.267081 12706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:25.274839 12706 catalog_manager.cc:1347] Generated new cluster ID: a9d865ba785a41faa9ee4645a766e3f7
I20260809 06:25:25.274950 12706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:25.290028 12706 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:25.291882 12706 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:25.303952 12706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16c8e5dd75a8410e8605babf728dde34: Generated new TSK 0
I20260809 06:25:25.304823 12706 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:25:25.366968 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:40407
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:25.703245 12724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:25.703514 12724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:25.703719 12724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:25.710963 12724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:25.711063 12724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:25.711230 12724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:25.720971 12724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:40407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:25.722266 12724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:25.723548 12724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:25.736644 12735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:25.736694 12734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:25.736732 12739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:25.737383 12724 server_base.cc:1034] running on GCE node
I20260809 06:25:25.737929 12724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:25.738765 12724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:25.739993 12724 hybrid_clock.cc:648] HybridClock initialized: now 1786256725739915 us; error 48 us; skew 500 ppm
I20260809 06:25:25.740510 12724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:25 dist-test-slave-mjjr krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256725, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:25.743942 12724 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:25:25.745362 12724 webserver.cc:460] Webserver started at http://127.8.131.1:40915/ using document root <none> and password file <none>
I20260809 06:25:25.746019 12724 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:25.746177 12724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:25.746520 12724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:25.749434 12724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "a16128ff283a4fd6aeaac5edb1524ea1"
format_stamp: "Formatted at 2026-08-09 06:25:25 on dist-test-slave-mjjr"
server_key: "1aba6e31f08491b226bfd11782089d31"
server_key_iv: "54f77e8bc261109ed03d5790fa08cb86"
server_key_version: "encryptionkey@0"
I20260809 06:25:25.750219 12724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "a16128ff283a4fd6aeaac5edb1524ea1"
format_stamp: "Formatted at 2026-08-09 06:25:25 on dist-test-slave-mjjr"
server_key: "1aba6e31f08491b226bfd11782089d31"
server_key_iv: "54f77e8bc261109ed03d5790fa08cb86"
server_key_version: "encryptionkey@0"
I20260809 06:25:25.755281 12724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:25:25.758889 12750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:25.760290 12724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:25.760528 12724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "a16128ff283a4fd6aeaac5edb1524ea1"
format_stamp: "Formatted at 2026-08-09 06:25:25 on dist-test-slave-mjjr"
server_key: "1aba6e31f08491b226bfd11782089d31"
server_key_iv: "54f77e8bc261109ed03d5790fa08cb86"
server_key_version: "encryptionkey@0"
I20260809 06:25:25.760715 12724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:25.794258 12724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:25.798768 12724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:25.799141 12724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:25.800225 12724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:25.801793 12724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:25.801868 12724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:25.802022 12724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:25.802104 12724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:25.836273 12724 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:45711
I20260809 06:25:25.836366 12933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:45711 every 8 connection(s)
I20260809 06:25:25.837611 12724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:25.838840 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 12724
I20260809 06:25:25.839107 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:25.839951 8716 external_mini_cluster.cc:1428] Setting key 3090441bdaaebb980c95fb3da822b71b
Aug 09 06:25:25 dist-test-slave-mjjr krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256725, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:25.858690 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:25.840742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:55647 (local address 127.8.131.62:40407)
0809 06:25:25.841270 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:25.841276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:25.842371 (+ 1095us) server_negotiation.cc:407] Connection header received
0809 06:25:25.844239 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:25.844244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:25.844335 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:25.844467 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:25.845921 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:25.846670 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:25.848350 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:25.848749 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:25.848860 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:25.852518 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:25.852523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:25.852527 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:25.852558 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:25.855028 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:25.855852 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:25.855859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:25.855864 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:25.856026 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:25.856527 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:25.856533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:25.856537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:25.856987 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:25.857178 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:25.857957 (+ 779us) server_negotiation.cc:299] Negotiation successful
0809 06:25:25.858224 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":266,"threads_started":1}
I20260809 06:25:25.859359 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:25.841353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40407 (local address 127.8.131.1:55647)
0809 06:25:25.842204 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:25.842260 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:25.843979 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:25.844698 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:25.844706 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:25.845212 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:25.845758 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:25.845774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:25.846839 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:25.846843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:25.848184 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:25.848203 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:25.849000 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:25.849009 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:25.849333 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:25.850112 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:25.850139 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:25.852292 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:25.855171 (+ 2879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:25.855180 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:25.855194 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:25.855695 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:25.856182 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:25.856192 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:25.856195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:25.856380 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:25.857128 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:25.857133 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:25.857491 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 06:25:25.857768 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 06:25:25.858142 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":332,"threads_started":1}
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
I20260809 06:25:25.861119 12934 heartbeater.cc:347] Connected to a master server at 127.8.131.62:40407
I20260809 06:25:25.861439 12934 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:25.862140 12934 heartbeater.cc:511] Master 127.8.131.62:40407 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
I20260809 06:25:25.864436 12635 ts_manager.cc:194] Registered new tserver with Master: a16128ff283a4fd6aeaac5edb1524ea1 (127.8.131.1:45711)
I20260809 06:25:25.866799 12635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:55647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:25:25.888511 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:40407
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:26.230554 12945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:26.230885 12945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:26.231066 12945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:26.238535 12945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:26.238742 12945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:26.238947 12945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:26.249325 12945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:40407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:26.250640 12945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:26.252000 12945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:26.266094 12958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:26.266171 12952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:26.266197 12951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:26.267158 12945 server_base.cc:1034] running on GCE node
I20260809 06:25:26.267823 12945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:26.268754 12945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:26.269975 12945 hybrid_clock.cc:648] HybridClock initialized: now 1786256726269927 us; error 54 us; skew 500 ppm
I20260809 06:25:26.270288 12945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:26 dist-test-slave-mjjr krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256726, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:26.273466 12945 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:25:26.274730 12945 webserver.cc:460] Webserver started at http://127.8.131.2:35597/ using document root <none> and password file <none>
I20260809 06:25:26.275445 12945 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:26.275596 12945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:26.276050 12945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:26.278899 12945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "9224228e7b0f4ffd97f1989eb33064a7"
format_stamp: "Formatted at 2026-08-09 06:25:26 on dist-test-slave-mjjr"
server_key: "42f88fcb6ee3cf9f083dc5e2660a3261"
server_key_iv: "7de1cbad0bc17c1fec7c4c7b14f04f95"
server_key_version: "encryptionkey@0"
I20260809 06:25:26.279616 12945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "9224228e7b0f4ffd97f1989eb33064a7"
format_stamp: "Formatted at 2026-08-09 06:25:26 on dist-test-slave-mjjr"
server_key: "42f88fcb6ee3cf9f083dc5e2660a3261"
server_key_iv: "7de1cbad0bc17c1fec7c4c7b14f04f95"
server_key_version: "encryptionkey@0"
I20260809 06:25:26.284581 12945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:25:26.287894 12972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:26.289134 12945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:26.289256 12945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "9224228e7b0f4ffd97f1989eb33064a7"
format_stamp: "Formatted at 2026-08-09 06:25:26 on dist-test-slave-mjjr"
server_key: "42f88fcb6ee3cf9f083dc5e2660a3261"
server_key_iv: "7de1cbad0bc17c1fec7c4c7b14f04f95"
server_key_version: "encryptionkey@0"
I20260809 06:25:26.289410 12945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:26.308573 12945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:26.312865 12945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:26.313110 12945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:26.314021 12945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:26.315465 12945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:26.315553 12945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:26.315663 12945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:26.315709 12945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 06:25:26.352175 12945 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:41043
I20260809 06:25:26.352222 13155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:41043 every 8 connection(s)
I20260809 06:25:26.353674 12945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:26.358801 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 12945
I20260809 06:25:26.359014 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:26.359557 8716 external_mini_cluster.cc:1428] Setting key 68d2a5e144c9e5b52217efc84c20184b
Aug 09 06:25:26 dist-test-slave-mjjr krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256726, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:26.374943 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:26.356463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:40113 (local address 127.8.131.62:40407)
0809 06:25:26.356659 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:26.356663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:26.358003 (+ 1340us) server_negotiation.cc:407] Connection header received
0809 06:25:26.360098 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:26.360104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:26.360202 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:26.360372 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:26.361980 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:26.362722 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:26.364584 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:26.365130 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:26.365288 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:26.369124 (+ 3836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:26.369130 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:26.369134 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:26.369168 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:26.371969 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:26.372698 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:26.372704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:26.372712 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:26.372852 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:26.373276 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:26.373281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:26.373289 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:26.373535 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:26.373695 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:26.374342 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 06:25:26.374560 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 06:25:26.376012 13161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:26.357040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40407 (local address 127.8.131.2:40113)
0809 06:25:26.357846 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:26.357883 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:26.359767 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:26.360653 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:26.360665 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:26.361197 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:26.361788 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:26.361807 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:26.362913 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:26.362919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:26.364421 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:26.364436 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:26.365384 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:26.365396 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:26.365715 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:26.366475 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:26.366515 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:26.368881 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:26.372113 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:26.372122 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:26.372136 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:26.372564 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:26.372967 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:26.372971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:26.372975 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:26.373152 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:26.373669 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:26.373676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:26.373988 (+ 312us) client_negotiation.cc:769] Sending connection context
0809 06:25:26.374240 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 06:25:26.374755 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":337,"threads_started":1}
I20260809 06:25:26.377470 13156 heartbeater.cc:347] Connected to a master server at 127.8.131.62:40407
I20260809 06:25:26.377774 13156 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:26.378448 13156 heartbeater.cc:511] Master 127.8.131.62:40407 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
I20260809 06:25:26.379850 12635 ts_manager.cc:194] Registered new tserver with Master: 9224228e7b0f4ffd97f1989eb33064a7 (127.8.131.2:41043)
I20260809 06:25:26.380775 12635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:40113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:25:26.406590 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:40407
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:26.739041 13166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:26.739360 13166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:26.739519 13166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:26.746850 13166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:26.746958 13166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:26.747171 13166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:26.756928 13166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:40407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:26.758193 13166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:26.759526 13166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:26.773445 13174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:26.773445 13180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:26.773510 13173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:26.774354 13166 server_base.cc:1034] running on GCE node
I20260809 06:25:26.775074 13166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:26.776127 13166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:26.777330 13166 hybrid_clock.cc:648] HybridClock initialized: now 1786256726777292 us; error 36 us; skew 500 ppm
I20260809 06:25:26.777654 13166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:26 dist-test-slave-mjjr krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256726, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:26.781042 13166 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:25:26.782572 13166 webserver.cc:460] Webserver started at http://127.8.131.3:45953/ using document root <none> and password file <none>
I20260809 06:25:26.783300 13166 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:26.783445 13166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:26.783857 13166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:26.786881 13166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "0a6da85d922041d58928b3460cab05f7"
format_stamp: "Formatted at 2026-08-09 06:25:26 on dist-test-slave-mjjr"
server_key: "ed1c40e4587025ddf4f8ee75c5ec5fde"
server_key_iv: "08047d76cdfffe52970c148d4b484e33"
server_key_version: "encryptionkey@0"
I20260809 06:25:26.787950 13166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "0a6da85d922041d58928b3460cab05f7"
format_stamp: "Formatted at 2026-08-09 06:25:26 on dist-test-slave-mjjr"
server_key: "ed1c40e4587025ddf4f8ee75c5ec5fde"
server_key_iv: "08047d76cdfffe52970c148d4b484e33"
server_key_version: "encryptionkey@0"
I20260809 06:25:26.793424 13166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:25:26.797070 13190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:26.798440 13166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:26.798596 13166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "0a6da85d922041d58928b3460cab05f7"
format_stamp: "Formatted at 2026-08-09 06:25:26 on dist-test-slave-mjjr"
server_key: "ed1c40e4587025ddf4f8ee75c5ec5fde"
server_key_iv: "08047d76cdfffe52970c148d4b484e33"
server_key_version: "encryptionkey@0"
I20260809 06:25:26.798777 13166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:26.813174 13166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:26.817425 13166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:26.817690 13166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:26.818632 13166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:26.820029 13166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:26.820101 13166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:26.820182 13166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:26.820230 13166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:26.854512 13166 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:45327
I20260809 06:25:26.854549 13366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:45327 every 8 connection(s)
I20260809 06:25:26.855980 13166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:26.856441 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 13166
I20260809 06:25:26.856680 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:26.857234 8716 external_mini_cluster.cc:1428] Setting key c7366ace725a0ff7ded2c45fefc675f4
Aug 09 06:25:26 dist-test-slave-mjjr krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256726, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:26.870666 12934 heartbeater.cc:503] Master 127.8.131.62:40407 was elected leader, sending a full tablet report...
I20260809 06:25:26.877311 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:26.859059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:34387 (local address 127.8.131.62:40407)
0809 06:25:26.859276 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:26.859282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:26.860926 (+ 1644us) server_negotiation.cc:407] Connection header received
0809 06:25:26.862728 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:26.862732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:26.862800 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:26.862884 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:26.864491 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:26.865314 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:26.867029 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:26.867469 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:26.867563 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:26.871035 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:26.871040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:26.871043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:26.871075 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:26.874351 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:26.875043 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:26.875047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:26.875049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:26.875175 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:26.875567 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:26.875570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:26.875572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:26.875913 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:26.876062 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:26.876701 (+ 639us) server_negotiation.cc:299] Negotiation successful
0809 06:25:26.876905 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 06:25:26.878070 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:26.859750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40407 (local address 127.8.131.3:34387)
0809 06:25:26.860700 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:26.860778 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:26.862502 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:26.863040 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:26.863047 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:26.863531 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:26.864332 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:26.864357 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:26.865454 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:26.865458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:26.866900 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:26.866912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:26.867758 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:26.867765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:26.867967 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:26.868567 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:26.868588 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:26.870799 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:26.874476 (+ 3677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:26.874483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:26.874494 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:26.874934 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:26.875268 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:26.875271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:26.875274 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:26.875455 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:26.876066 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:26.876072 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:26.876372 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 06:25:26.876591 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 06:25:26.876989 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":317,"threads_started":1}
I20260809 06:25:26.879352 13371 heartbeater.cc:347] Connected to a master server at 127.8.131.62:40407
I20260809 06:25:26.879693 13371 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:26.880420 13371 heartbeater.cc:511] Master 127.8.131.62:40407 requested a full tablet report, sending...
I20260809 06:25:26.881757 12635 ts_manager.cc:194] Registered new tserver with Master: 0a6da85d922041d58928b3460cab05f7 (127.8.131.3:45327)
I20260809 06:25:26.882519 12635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:34387
I20260809 06:25:26.885607 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 06:25:26.920718 12936 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:25:26.918008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.8.131.62:40407)
0809 06:25:26.918274 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:26.918283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:26.918622 (+ 339us) server_negotiation.cc:407] Connection header received
0809 06:25:26.918851 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:26.918855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:26.918930 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:26.919046 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:26.920359 (+ 1313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58442: BlockingRecv error: recv got EOF from 127.0.0.1:58442 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 06:25:26.924053 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 12724
W20260809 06:25:26.926887 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 12724: Not authorized: Client connection negotiation failed: client connection to 127.8.131.1:45711: server requires authentication, but client does not have Kerberos credentials available
I20260809 06:25:26.927085 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 12724
I20260809 06:25:26.945781 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 12945
W20260809 06:25:26.948551 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 12945: Not authorized: Client connection negotiation failed: client connection to 127.8.131.2:41043: server requires authentication, but client does not have Kerberos credentials available
I20260809 06:25:26.948745 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 12945
I20260809 06:25:26.972451 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 13166
W20260809 06:25:26.975570 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 13166: Not authorized: Client connection negotiation failed: client connection to 127.8.131.3:45327: server requires authentication, but client does not have Kerberos credentials available
I20260809 06:25:26.975831 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 13166
I20260809 06:25:26.996852 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 12586
I20260809 06:25:26.998215 12677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:27.100931 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 12586
2026-08-09T06:25:27Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3843 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:27 dist-test-slave-mjjr krb5kdc[13398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:27 dist-test-slave-mjjr krb5kdc[13398](info): set up 2 sockets
Aug 09 06:25:27 dist-test-slave-mjjr krb5kdc[13398](info): commencing operation
krb5kdc: starting...
W20260809 06:25:28.389173 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:28 dist-test-slave-mjjr krb5kdc[13398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:28.599387 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:42175
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:44273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:42175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:28.936232 13422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:28.936533 13422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:28.936625 13422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:28.943988 13422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:28.944099 13422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:28.944147 13422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:28.944208 13422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:28.944263 13422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:28.954214 13422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:42175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:42175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:28.955458 13422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:28.956744 13422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:28.967968 13428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:28.968107 13427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:28.969518 13431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:28.970044 13422 server_base.cc:1034] running on GCE node
I20260809 06:25:28.970938 13422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:28.972512 13422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:28.973781 13422 hybrid_clock.cc:648] HybridClock initialized: now 1786256728973705 us; error 49 us; skew 500 ppm
I20260809 06:25:28.974174 13422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:28 dist-test-slave-mjjr krb5kdc[13398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256728, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:28.982309 13422 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:28.983994 13422 webserver.cc:460] Webserver started at http://127.8.131.62:42683/ using document root <none> and password file <none>
I20260809 06:25:28.984720 13422 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:28.984889 13422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:28.985219 13422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:28.988430 13422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "215ff33f4e144209a39d512217f5dc58"
format_stamp: "Formatted at 2026-08-09 06:25:28 on dist-test-slave-mjjr"
server_key: "b36ef2676553f6c1f7436f42a1afae40"
server_key_iv: "7fa9dd2d724568be2ba2e90429e16519"
server_key_version: "encryptionkey@0"
I20260809 06:25:28.989272 13422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "215ff33f4e144209a39d512217f5dc58"
format_stamp: "Formatted at 2026-08-09 06:25:28 on dist-test-slave-mjjr"
server_key: "b36ef2676553f6c1f7436f42a1afae40"
server_key_iv: "7fa9dd2d724568be2ba2e90429e16519"
server_key_version: "encryptionkey@0"
I20260809 06:25:28.994593 13422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:25:28.998098 13440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:28.999408 13422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:25:28.999609 13422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "215ff33f4e144209a39d512217f5dc58"
format_stamp: "Formatted at 2026-08-09 06:25:28 on dist-test-slave-mjjr"
server_key: "b36ef2676553f6c1f7436f42a1afae40"
server_key_iv: "7fa9dd2d724568be2ba2e90429e16519"
server_key_version: "encryptionkey@0"
I20260809 06:25:28.999876 13422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:29.022183 13422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:29.026448 13422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:29.026791 13422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:29.045995 13422 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:42175
I20260809 06:25:29.046028 13521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:42175 every 8 connection(s)
I20260809 06:25:29.047472 13422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:29.051244 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 13422
I20260809 06:25:29.051338 13523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:29.051472 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:29.052361 8716 external_mini_cluster.cc:1428] Setting key 9944d84d4f79dcebdd6945688b85846a
I20260809 06:25:29.064024 13523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58: Bootstrap starting.
Aug 09 06:25:29 dist-test-slave-mjjr krb5kdc[13398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:29.067612 13523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:29.068856 13523 log.cc:824] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:29.072105 13523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58: No bootstrap required, opened a new log
I20260809 06:25:29.074627 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:29.054373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.8.131.62:42175)
0809 06:25:29.055190 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:29.055201 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:29.055240 (+ 39us) server_negotiation.cc:407] Connection header received
0809 06:25:29.056778 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:29.056797 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:29.057164 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:29.057567 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:29.059031 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:29.060398 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:29.061661 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:29.062347 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:29.062558 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:29.067034 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:29.067041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:29.067054 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:29.067094 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:29.070536 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:29.071344 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:29.071350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:29.071355 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:29.071508 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:29.071966 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:29.071969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:29.071971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:29.072541 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:29.072780 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:29.073250 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 06:25:29.073576 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":387,"threads_started":1}
I20260809 06:25:29.078471 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:25:29.080404 13523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215ff33f4e144209a39d512217f5dc58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42175 } }
I20260809 06:25:29.080744 13523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:29.080862 13523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215ff33f4e144209a39d512217f5dc58, State: Initialized, Role: FOLLOWER
I20260809 06:25:29.081421 13523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215ff33f4e144209a39d512217f5dc58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42175 } }
I20260809 06:25:29.081647 13523 raft_consensus.cc:388] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:29.081775 13523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:29.081955 13523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:29.084134 13523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215ff33f4e144209a39d512217f5dc58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42175 } }
I20260809 06:25:29.084683 13523 leader_election.cc:302] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 215ff33f4e144209a39d512217f5dc58; no voters:
I20260809 06:25:29.085489 13523 leader_election.cc:288] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:29.085680 13534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:29.086951 13534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [term 1 LEADER]: Becoming Leader. State: Replica: 215ff33f4e144209a39d512217f5dc58, State: Running, Role: LEADER
I20260809 06:25:29.087514 13534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215ff33f4e144209a39d512217f5dc58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42175 } }
I20260809 06:25:29.087903 13523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:29.096159 13536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 215ff33f4e144209a39d512217f5dc58. Latest consensus state: current_term: 1 leader_uuid: "215ff33f4e144209a39d512217f5dc58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215ff33f4e144209a39d512217f5dc58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42175 } } }
I20260809 06:25:29.096558 13536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:29.096835 13535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "215ff33f4e144209a39d512217f5dc58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215ff33f4e144209a39d512217f5dc58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42175 } } }
I20260809 06:25:29.097167 13535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:29.097882 13565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:29.102542 13565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:29.111836 13565 catalog_manager.cc:1347] Generated new cluster ID: d9fb4d2f4d1043cc827e701ff02c2c0d
I20260809 06:25:29.111955 13565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:29.127235 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:29.119545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38018 (local address 127.8.131.62:42175)
0809 06:25:29.119793 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:29.119802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:29.120408 (+ 606us) server_negotiation.cc:407] Connection header received
0809 06:25:29.120616 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:29.120622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:29.120697 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:29.120827 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:29.121849 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:29.122594 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:29.123954 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:29.124406 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:29.124555 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:29.125669 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:29.125674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:29.125677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:29.126392 (+ 715us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:29.126434 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:29.126477 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:29.126607 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:29.126748 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 06:25:29.126852 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 06:25:29.137844 13565 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:29.139394 13565 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:29.151445 13565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 215ff33f4e144209a39d512217f5dc58: Generated new TSK 0
I20260809 06:25:29.152367 13565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:25:29.160483 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 13422
I20260809 06:25:29.161826 13514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:29.252628 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 13422
2026-08-09T06:25:29Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2148 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:29 dist-test-slave-mjjr krb5kdc[13585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:29 dist-test-slave-mjjr krb5kdc[13585](info): set up 2 sockets
Aug 09 06:25:29 dist-test-slave-mjjr krb5kdc[13585](info): commencing operation
krb5kdc: starting...
W20260809 06:25:30.567991 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:30 dist-test-slave-mjjr krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:30.784336 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:32959
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:32959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:31.112174 13608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:31.112464 13608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:31.112666 13608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:31.119899 13608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:31.120007 13608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:31.120132 13608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:31.120236 13608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:31.120302 13608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:31.130122 13608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:32959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:32959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:31.131558 13608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:31.132970 13608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:31.145191 13614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:31.145196 13616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:31.145283 13619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:31.146251 13608 server_base.cc:1034] running on GCE node
I20260809 06:25:31.147053 13608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:31.148558 13608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:31.149799 13608 hybrid_clock.cc:648] HybridClock initialized: now 1786256731149757 us; error 55 us; skew 500 ppm
I20260809 06:25:31.150153 13608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:31 dist-test-slave-mjjr krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256731, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:31.153426 13608 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:31.155022 13608 webserver.cc:460] Webserver started at http://127.8.131.62:38805/ using document root <none> and password file <none>
I20260809 06:25:31.155777 13608 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:31.155933 13608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:31.156252 13608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:31.159281 13608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "60a65f9728cf462ca64bb9a1b600ba58"
format_stamp: "Formatted at 2026-08-09 06:25:31 on dist-test-slave-mjjr"
server_key: "064e68db776e8916fb85df955f13ecfc"
server_key_iv: "e14497bbf58d510105288dfdfb2aa543"
server_key_version: "encryptionkey@0"
I20260809 06:25:31.160102 13608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "60a65f9728cf462ca64bb9a1b600ba58"
format_stamp: "Formatted at 2026-08-09 06:25:31 on dist-test-slave-mjjr"
server_key: "064e68db776e8916fb85df955f13ecfc"
server_key_iv: "e14497bbf58d510105288dfdfb2aa543"
server_key_version: "encryptionkey@0"
I20260809 06:25:31.165540 13608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:31.169191 13629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:31.170608 13608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:31.170804 13608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "60a65f9728cf462ca64bb9a1b600ba58"
format_stamp: "Formatted at 2026-08-09 06:25:31 on dist-test-slave-mjjr"
server_key: "064e68db776e8916fb85df955f13ecfc"
server_key_iv: "e14497bbf58d510105288dfdfb2aa543"
server_key_version: "encryptionkey@0"
I20260809 06:25:31.171018 13608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:31.196031 13608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:31.200161 13608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:31.200461 13608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:31.219372 13608 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:32959
I20260809 06:25:31.219411 13710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:32959 every 8 connection(s)
I20260809 06:25:31.220753 13608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:31.224512 13713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:31.225147 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 13608
I20260809 06:25:31.225378 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:31.226042 8716 external_mini_cluster.cc:1428] Setting key 2c6442f15d44a33cd1aff5bf7539c6d6
I20260809 06:25:31.239039 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58: Bootstrap starting.
Aug 09 06:25:31 dist-test-slave-mjjr krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:31.242522 13713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:31.243970 13713 log.cc:824] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:31.247502 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58: No bootstrap required, opened a new log
I20260809 06:25:31.251233 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:31.228244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.8.131.62:32959)
0809 06:25:31.229088 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:31.229104 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:31.229161 (+ 57us) server_negotiation.cc:407] Connection header received
0809 06:25:31.231195 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:31.231223 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:31.231769 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:31.232240 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:31.233526 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:31.235560 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:31.236829 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:31.237871 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:31.238048 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:31.242449 (+ 4401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:31.242461 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:31.242481 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:31.242540 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:31.246423 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:31.247108 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:31.247117 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:31.247125 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:31.247330 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:31.247791 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:31.247795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:31.247798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:31.248554 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:31.248902 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:31.249199 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 06:25:31.249653 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":331,"threads_started":1}
I20260809 06:25:31.256489 13713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a65f9728cf462ca64bb9a1b600ba58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 32959 } }
I20260809 06:25:31.256863 13713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:31.256989 13713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a65f9728cf462ca64bb9a1b600ba58, State: Initialized, Role: FOLLOWER
I20260809 06:25:31.257545 13713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a65f9728cf462ca64bb9a1b600ba58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 32959 } }
I20260809 06:25:31.257771 13713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:31.257932 13713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:31.258107 13713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:31.260452 13713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a65f9728cf462ca64bb9a1b600ba58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 32959 } }
I20260809 06:25:31.260908 13713 leader_election.cc:302] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60a65f9728cf462ca64bb9a1b600ba58; no voters:
I20260809 06:25:31.261482 13713 leader_election.cc:288] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:31.261633 13724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:31.262324 13724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [term 1 LEADER]: Becoming Leader. State: Replica: 60a65f9728cf462ca64bb9a1b600ba58, State: Running, Role: LEADER
I20260809 06:25:31.262879 13724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a65f9728cf462ca64bb9a1b600ba58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 32959 } }
I20260809 06:25:31.263604 13713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:31.265746 13726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60a65f9728cf462ca64bb9a1b600ba58. Latest consensus state: current_term: 1 leader_uuid: "60a65f9728cf462ca64bb9a1b600ba58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a65f9728cf462ca64bb9a1b600ba58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 32959 } } }
I20260809 06:25:31.266106 13726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:31.265879 13725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60a65f9728cf462ca64bb9a1b600ba58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a65f9728cf462ca64bb9a1b600ba58" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 32959 } } }
I20260809 06:25:31.266299 13725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:31.273048 13733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:31.278275 13733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:31.287087 13733 catalog_manager.cc:1347] Generated new cluster ID: b18a8026d2334e41ab1d365eec9421c0
I20260809 06:25:31.287204 13733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:31.304522 13733 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:31.305855 13733 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:31.312559 13733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60a65f9728cf462ca64bb9a1b600ba58: Generated new TSK 0
I20260809 06:25:31.313403 13733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:25:31.369357 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:32959
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:31.707172 13760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:31.707494 13760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:31.707736 13760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:31.715315 13760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:31.715433 13760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:31.715605 13760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:31.725772 13760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:32959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:31.727243 13760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:31.728696 13760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:31.742120 13771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:31.742573 13775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:31.742614 13772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:31.744098 13760 server_base.cc:1034] running on GCE node
I20260809 06:25:31.744781 13760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:31.745841 13760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:31.747112 13760 hybrid_clock.cc:648] HybridClock initialized: now 1786256731747045 us; error 60 us; skew 500 ppm
I20260809 06:25:31.747476 13760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:31 dist-test-slave-mjjr krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256731, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:31.750902 13760 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:25:31.752568 13760 webserver.cc:460] Webserver started at http://127.8.131.1:36783/ using document root <none> and password file <none>
I20260809 06:25:31.753315 13760 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:31.753463 13760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:31.753777 13760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:31.756958 13760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "1869a0fc959b46f4aaf8357069ba65eb"
format_stamp: "Formatted at 2026-08-09 06:25:31 on dist-test-slave-mjjr"
server_key: "24c15c18144147cd4d51bd53ecba7007"
server_key_iv: "0c6f49b3646c7930f58a090fcafff88b"
server_key_version: "encryptionkey@0"
I20260809 06:25:31.757756 13760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "1869a0fc959b46f4aaf8357069ba65eb"
format_stamp: "Formatted at 2026-08-09 06:25:31 on dist-test-slave-mjjr"
server_key: "24c15c18144147cd4d51bd53ecba7007"
server_key_iv: "0c6f49b3646c7930f58a090fcafff88b"
server_key_version: "encryptionkey@0"
I20260809 06:25:31.762913 13760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:25:31.766701 13783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:31.768188 13760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:31.768373 13760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "1869a0fc959b46f4aaf8357069ba65eb"
format_stamp: "Formatted at 2026-08-09 06:25:31 on dist-test-slave-mjjr"
server_key: "24c15c18144147cd4d51bd53ecba7007"
server_key_iv: "0c6f49b3646c7930f58a090fcafff88b"
server_key_version: "encryptionkey@0"
I20260809 06:25:31.768581 13760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:31.791544 13760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:31.796092 13760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:31.796432 13760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:31.797446 13760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:31.798903 13760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:31.799018 13760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:31.799170 13760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:31.799244 13760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:31.834564 13760 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:36893
I20260809 06:25:31.834663 13977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:36893 every 8 connection(s)
I20260809 06:25:31.835988 13760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:31.844187 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 13760
I20260809 06:25:31.844422 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:31.845081 8716 external_mini_cluster.cc:1428] Setting key 0eeb76323e6b6de7677b9779c6905a2d
Aug 09 06:25:31 dist-test-slave-mjjr krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256731, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:31.859444 13983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:31.838671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:37645 (local address 127.8.131.62:32959)
0809 06:25:31.839176 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:31.839181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:31.840077 (+ 896us) server_negotiation.cc:407] Connection header received
0809 06:25:31.842084 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:31.842091 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:31.842191 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:31.842331 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:31.844144 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:31.845047 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:31.847005 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:31.847489 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:31.847601 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:31.852246 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:31.852256 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:31.852264 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:31.852317 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:31.855813 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:31.856862 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:31.856867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:31.856869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:31.857011 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:31.857558 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:31.857562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:31.857564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:31.857885 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:31.858010 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:31.858990 (+ 980us) server_negotiation.cc:299] Negotiation successful
0809 06:25:31.859171 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":283,"threads_started":1}
I20260809 06:25:31.861820 13985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:31.839053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:32959 (local address 127.8.131.1:37645)
0809 06:25:31.839903 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:31.839945 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:31.841784 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:31.842596 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:31.842604 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:31.843128 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:31.843903 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:31.843944 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:31.845234 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:31.845240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:31.846834 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:31.846852 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:31.847781 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:31.847792 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:31.848142 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:31.848950 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:31.848993 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:31.851973 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:31.855964 (+ 3991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:31.855977 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:31.855997 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:31.856697 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:31.857144 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:31.857152 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:31.857159 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:31.857422 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:31.858024 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:31.858035 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:31.858515 (+ 480us) client_negotiation.cc:769] Sending connection context
0809 06:25:31.858833 (+ 318us) client_negotiation.cc:240] Negotiation successful
0809 06:25:31.859489 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":319,"threads_started":1}
I20260809 06:25:31.863686 13980 heartbeater.cc:347] Connected to a master server at 127.8.131.62:32959
I20260809 06:25:31.864003 13980 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:31.865111 13980 heartbeater.cc:511] Master 127.8.131.62:32959 requested a full tablet report, sending...
I20260809 06:25:31.868073 13658 ts_manager.cc:194] Registered new tserver with Master: 1869a0fc959b46f4aaf8357069ba65eb (127.8.131.1:36893)
I20260809 06:25:31.870223 13658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:37645
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:25:31.899258 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:32959
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:32.226835 13988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:32.227157 13988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:32.227377 13988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:32.234656 13988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:32.234843 13988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:32.235030 13988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:32.244822 13988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:32959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:32.246155 13988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:32.247495 13988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:32.260844 13996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:32.260905 13997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:32.260937 14000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:32.262292 13988 server_base.cc:1034] running on GCE node
I20260809 06:25:32.262910 13988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:32.263958 13988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:32.265187 13988 hybrid_clock.cc:648] HybridClock initialized: now 1786256732265136 us; error 64 us; skew 500 ppm
I20260809 06:25:32.265493 13988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:32 dist-test-slave-mjjr krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256732, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:32.268756 13988 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:25:32.270159 13988 webserver.cc:460] Webserver started at http://127.8.131.2:38093/ using document root <none> and password file <none>
I20260809 06:25:32.270861 13988 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:32.271013 13988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:32.271335 13988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:32.274667 13988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "3e6a84cb19d0445dab2b510a81457c91"
format_stamp: "Formatted at 2026-08-09 06:25:32 on dist-test-slave-mjjr"
server_key: "722d15c60d0e98d6cf9e9696cdd32c57"
server_key_iv: "b6497ce3d569fb6581eb0a89f534fd0c"
server_key_version: "encryptionkey@0"
I20260809 06:25:32.275534 13988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "3e6a84cb19d0445dab2b510a81457c91"
format_stamp: "Formatted at 2026-08-09 06:25:32 on dist-test-slave-mjjr"
server_key: "722d15c60d0e98d6cf9e9696cdd32c57"
server_key_iv: "b6497ce3d569fb6581eb0a89f534fd0c"
server_key_version: "encryptionkey@0"
I20260809 06:25:32.280685 13988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:25:32.284153 14015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:32.285418 13988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 06:25:32.285591 13988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "3e6a84cb19d0445dab2b510a81457c91"
format_stamp: "Formatted at 2026-08-09 06:25:32 on dist-test-slave-mjjr"
server_key: "722d15c60d0e98d6cf9e9696cdd32c57"
server_key_iv: "b6497ce3d569fb6581eb0a89f534fd0c"
server_key_version: "encryptionkey@0"
I20260809 06:25:32.285799 13988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:32.315843 13988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:32.320314 13988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:32.320640 13988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:32.321614 13988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:32.323050 13988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:32.323158 13988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:32.323308 13988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:32.323395 13988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:32.356886 13988 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:33495
I20260809 06:25:32.356938 14202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:33495 every 8 connection(s)
I20260809 06:25:32.358177 13988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:32.360033 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 13988
I20260809 06:25:32.360246 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:32.360859 8716 external_mini_cluster.cc:1428] Setting key 58073fec2724b2fce5b4bcbce7f9067d
Aug 09 06:25:32 dist-test-slave-mjjr krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256732, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:32.382743 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:32.360793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:34143 (local address 127.8.131.62:32959)
0809 06:25:32.361340 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:32.361344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:32.362443 (+ 1099us) server_negotiation.cc:407] Connection header received
0809 06:25:32.365160 (+ 2717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:32.365164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:32.365227 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:32.365329 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:32.367398 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.368219 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.370721 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.371473 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.371603 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:32.375283 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:32.375292 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:32.375300 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:32.375344 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:32.379190 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:32.379990 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:32.379998 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:32.380004 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:32.380166 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:32.380757 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:32.380764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:32.380767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:32.381158 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:32.381347 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:32.381984 (+ 637us) server_negotiation.cc:299] Negotiation successful
0809 06:25:32.382263 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":315,"threads_started":1}
I20260809 06:25:32.383265 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:32.361431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:32959 (local address 127.8.131.2:34143)
0809 06:25:32.362258 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:32.362303 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:32.364837 (+ 2534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:32.365600 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:32.365613 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:32.366367 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:32.367191 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:32.367218 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.368399 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.368407 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:32.370531 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:32.370549 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.371754 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.371760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:32.371953 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:32.372699 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:32.372721 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:32.375050 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:32.379365 (+ 4315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:32.379371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:32.379408 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:32.379872 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:32.380280 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:32.380283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:32.380285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:32.380615 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:32.381292 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:32.381297 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:32.381603 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 06:25:32.381811 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 06:25:32.382174 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":379,"threads_started":1}
I20260809 06:25:32.384806 14208 heartbeater.cc:347] Connected to a master server at 127.8.131.62:32959
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
I20260809 06:25:32.385131 14208 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:32.385895 14208 heartbeater.cc:511] Master 127.8.131.62:32959 requested a full tablet report, sending...
I20260809 06:25:32.387332 13658 ts_manager.cc:194] Registered new tserver with Master: 3e6a84cb19d0445dab2b510a81457c91 (127.8.131.2:33495)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
I20260809 06:25:32.388266 13658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:34143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:25:32.414691 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:32959
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:32.748203 14217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:32.748544 14217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:32.748713 14217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:32.756233 14217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:32.756407 14217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:32.756585 14217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:32.766731 14217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:32959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:32.768167 14217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:32.769492 14217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:32.783824 14228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:32.783898 14231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:32.783949 14226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:32.785029 14217 server_base.cc:1034] running on GCE node
I20260809 06:25:32.785682 14217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:32.786659 14217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:32.787933 14217 hybrid_clock.cc:648] HybridClock initialized: now 1786256732787891 us; error 51 us; skew 500 ppm
I20260809 06:25:32.788328 14217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:32 dist-test-slave-mjjr krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256732, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:32.791756 14217 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:25:32.793327 14217 webserver.cc:460] Webserver started at http://127.8.131.3:40183/ using document root <none> and password file <none>
I20260809 06:25:32.794050 14217 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:32.794200 14217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:32.794526 14217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:32.797725 14217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "ae6612b9eed34b339a23be5280d82483"
format_stamp: "Formatted at 2026-08-09 06:25:32 on dist-test-slave-mjjr"
server_key: "e64c80dc6cd1bd6d9426188498adee7b"
server_key_iv: "d5858fba15c62e847fc8f7dd25101df4"
server_key_version: "encryptionkey@0"
I20260809 06:25:32.798580 14217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "ae6612b9eed34b339a23be5280d82483"
format_stamp: "Formatted at 2026-08-09 06:25:32 on dist-test-slave-mjjr"
server_key: "e64c80dc6cd1bd6d9426188498adee7b"
server_key_iv: "d5858fba15c62e847fc8f7dd25101df4"
server_key_version: "encryptionkey@0"
I20260809 06:25:32.803907 14217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:25:32.807538 14241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:32.808945 14217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:32.809139 14217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "ae6612b9eed34b339a23be5280d82483"
format_stamp: "Formatted at 2026-08-09 06:25:32 on dist-test-slave-mjjr"
server_key: "e64c80dc6cd1bd6d9426188498adee7b"
server_key_iv: "d5858fba15c62e847fc8f7dd25101df4"
server_key_version: "encryptionkey@0"
I20260809 06:25:32.809360 14217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:32.822281 14217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:32.826298 14217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:32.826628 14217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:32.827616 14217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:32.829033 14217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:32.829141 14217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:32.829293 14217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:32.829373 14217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:32.864216 14217 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:42729
I20260809 06:25:32.864306 14440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:42729 every 8 connection(s)
I20260809 06:25:32.865636 14217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:32.866550 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 14217
I20260809 06:25:32.866739 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:32.867311 8716 external_mini_cluster.cc:1428] Setting key cc66aaf646fb9747be0c32aeb287c451
I20260809 06:25:32.874131 13980 heartbeater.cc:503] Master 127.8.131.62:32959 was elected leader, sending a full tablet report...
Aug 09 06:25:32 dist-test-slave-mjjr krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256732, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:32.886737 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:32.868533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:34413 (local address 127.8.131.62:32959)
0809 06:25:32.868747 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:32.868753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:32.870149 (+ 1396us) server_negotiation.cc:407] Connection header received
0809 06:25:32.872336 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:32.872340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:32.872418 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:32.872503 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:32.874277 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.875221 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.876928 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.877328 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.877426 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:32.880879 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:32.880883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:32.880887 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:32.880917 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:32.883614 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:32.884400 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:32.884405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:32.884409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:32.884533 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:32.884984 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:32.884990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:32.884993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:32.885261 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:32.885446 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:32.886153 (+ 707us) server_negotiation.cc:299] Negotiation successful
0809 06:25:32.886387 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:25:32.887749 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:32.869018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:32959 (local address 127.8.131.3:34413)
0809 06:25:32.869966 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:32.870018 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:32.872086 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:32.872817 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:32.872825 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:32.873412 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:32.874110 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:32.874128 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.875379 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.875385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:32.876793 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:32.876811 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.877515 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.877520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:32.877709 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:32.878433 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:32.878456 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:32.880693 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:32.883807 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:32.883824 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:32.883841 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:32.884263 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:32.884647 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:32.884652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:32.884655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:32.884860 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:32.885401 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:32.885410 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:32.885745 (+ 335us) client_negotiation.cc:769] Sending connection context
0809 06:25:32.886028 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 06:25:32.886580 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":397,"threads_started":1}
I20260809 06:25:32.889029 14441 heartbeater.cc:347] Connected to a master server at 127.8.131.62:32959
I20260809 06:25:32.889385 14441 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:32.890132 14441 heartbeater.cc:511] Master 127.8.131.62:32959 requested a full tablet report, sending...
I20260809 06:25:32.891603 13658 ts_manager.cc:194] Registered new tserver with Master: ae6612b9eed34b339a23be5280d82483 (127.8.131.3:42729)
I20260809 06:25:32.892443 13658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:34413
I20260809 06:25:32.895388 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 06:25:32 dist-test-slave-mjjr krb5kdc[13585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256732, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 06:25:32 dist-test-slave-mjjr krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256732, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:32.947184 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:32.933483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.8.131.62:32959)
0809 06:25:32.933670 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:32.933675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:32.934166 (+ 491us) server_negotiation.cc:407] Connection header received
0809 06:25:32.934324 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:32.934328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:32.934410 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:32.934505 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:32.935892 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.936656 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.937863 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.938267 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.938360 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:32.942257 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:32.942262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:32.942265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:32.942299 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:32.944981 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:32.945567 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:32.945571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:32.945573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:32.945689 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:32.946012 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:32.946016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:32.946018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:32.946331 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:32.946553 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:32.946767 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 06:25:32.946934 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 06:25:32.947855 13658 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58966
I20260809 06:25:32.949424 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 13760
Aug 09 06:25:32 dist-test-slave-mjjr krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256732, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:32.966792 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:32.950815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.8.131.1:36893)
0809 06:25:32.951500 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:32.951507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:32.951533 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:25:32.951588 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:32.951595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:32.951879 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:32.952024 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:32.953154 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.954175 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.955335 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.956090 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.956195 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:32.960586 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:32.960603 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:32.960620 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:32.960663 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:32.963876 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:32.964782 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:32.964790 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:32.964793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:32.964933 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:32.965407 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:32.965411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:32.965413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:32.965765 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:32.966009 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:32.966263 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 06:25:32.966494 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":358,"threads_started":1}
W20260809 06:25:32.968163 13970 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51448
W20260809 06:25:32.969324 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 13760: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:25:32.969542 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 13760
I20260809 06:25:32.994092 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 13988
Aug 09 06:25:33 dist-test-slave-mjjr krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256732, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:25:33.010689 14465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:32.995090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55656 (local address 127.8.131.2:33495)
0809 06:25:32.995715 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:32.995722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:32.995748 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:25:32.995808 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:32.995814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:32.996052 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:32.996227 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:32.997513 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:32.998518 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:32.999845 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:33.000421 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:33.000577 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:33.004821 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:33.004832 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:33.004845 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:33.004881 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:33.007761 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:33.008448 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:33.008458 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:33.008463 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:33.008594 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:33.008996 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:33.009001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:33.009005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:33.009358 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:33.009709 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:33.010027 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 06:25:33.010293 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":288,"threads_started":1}
W20260809 06:25:33.012112 14194 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55656
W20260809 06:25:33.013314 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 13988: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:25:33.013607 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 13988
I20260809 06:25:33.036019 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 14217
Aug 09 06:25:33 dist-test-slave-mjjr krb5kdc[13585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256732, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:25:33.052517 14466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:33.036975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.8.131.3:42729)
0809 06:25:33.037560 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:33.037568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:33.037597 (+ 29us) server_negotiation.cc:407] Connection header received
0809 06:25:33.037657 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:33.037663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:33.037876 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:33.037999 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:33.039318 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:33.040448 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:33.041676 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:33.042231 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:33.042337 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:33.046856 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:33.046867 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:33.046885 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:33.046934 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:33.049937 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:33.050589 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:33.050596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:33.050599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:33.050731 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:33.051056 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:33.051059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:33.051061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:33.051413 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:33.051708 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:33.052000 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 06:25:33.052208 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":261,"threads_started":1}
W20260809 06:25:33.053836 14432 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36584
W20260809 06:25:33.055013 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 14217: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:25:33.055213 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 14217
I20260809 06:25:33.076968 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 13608
I20260809 06:25:33.078259 13705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:33.183686 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 13608
2026-08-09T06:25:33Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3934 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:33 dist-test-slave-mjjr krb5kdc[14472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:33 dist-test-slave-mjjr krb5kdc[14472](info): set up 2 sockets
Aug 09 06:25:33 dist-test-slave-mjjr krb5kdc[14472](info): commencing operation
krb5kdc: starting...
W20260809 06:25:34.488168 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:34 dist-test-slave-mjjr krb5kdc[14472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:34.707176 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45795
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:45795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:35.042255 14490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:35.042575 14490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:35.042752 14490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:35.050086 14490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:35.050210 14490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:35.050311 14490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:35.050402 14490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:35.050478 14490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:35.060503 14490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:45795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:35.061924 14490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:35.063314 14490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:35.075502 14498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:35.075546 14503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:35.075584 14497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:35.076507 14490 server_base.cc:1034] running on GCE node
I20260809 06:25:35.077374 14490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:35.078814 14490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:35.080024 14490 hybrid_clock.cc:648] HybridClock initialized: now 1786256735079963 us; error 53 us; skew 500 ppm
I20260809 06:25:35.080396 14490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:35 dist-test-slave-mjjr krb5kdc[14472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256735, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:35.088395 14490 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:35.089877 14490 webserver.cc:460] Webserver started at http://127.8.131.62:41825/ using document root <none> and password file <none>
I20260809 06:25:35.090534 14490 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:35.090641 14490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:35.090909 14490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:35.094058 14490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "220a7c74a9a248a4b9586ec65834e898"
format_stamp: "Formatted at 2026-08-09 06:25:35 on dist-test-slave-mjjr"
server_key: "ad795ffa2a0dd62b0b455b22528b5095"
server_key_iv: "1c2c45b81ad08b66904172c0cbd9ee6f"
server_key_version: "encryptionkey@0"
I20260809 06:25:35.094798 14490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "220a7c74a9a248a4b9586ec65834e898"
format_stamp: "Formatted at 2026-08-09 06:25:35 on dist-test-slave-mjjr"
server_key: "ad795ffa2a0dd62b0b455b22528b5095"
server_key_iv: "1c2c45b81ad08b66904172c0cbd9ee6f"
server_key_version: "encryptionkey@0"
I20260809 06:25:35.099996 14490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 06:25:35.103319 14517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:35.104708 14490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:35.104897 14490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "220a7c74a9a248a4b9586ec65834e898"
format_stamp: "Formatted at 2026-08-09 06:25:35 on dist-test-slave-mjjr"
server_key: "ad795ffa2a0dd62b0b455b22528b5095"
server_key_iv: "1c2c45b81ad08b66904172c0cbd9ee6f"
server_key_version: "encryptionkey@0"
I20260809 06:25:35.105108 14490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:35.127488 14490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:35.132113 14490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:35.132429 14490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:35.151345 14490 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:45795
I20260809 06:25:35.151423 14618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:45795 every 8 connection(s)
I20260809 06:25:35.152765 14490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:35.156718 14621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:35.158901 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 14490
I20260809 06:25:35.159143 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:35.159996 8716 external_mini_cluster.cc:1428] Setting key 875375d00027fc01216f710878a17abf
I20260809 06:25:35.172655 14621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898: Bootstrap starting.
I20260809 06:25:35.176913 14621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:35.178190 14621 log.cc:824] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898: Log is configured to *not* fsync() on all Append() calls
Aug 09 06:25:35 dist-test-slave-mjjr krb5kdc[14472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:35.181299 14621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898: No bootstrap required, opened a new log
I20260809 06:25:35.187405 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:35.162210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.8.131.62:45795)
0809 06:25:35.163367 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:35.163384 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:35.163457 (+ 73us) server_negotiation.cc:407] Connection header received
0809 06:25:35.166375 (+ 2918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:35.166410 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:35.167055 (+ 645us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:35.167531 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:35.169479 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:35.171567 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:35.173401 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:35.174379 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:35.174528 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:35.179510 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:35.179523 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:35.179547 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:35.179584 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:35.182762 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:35.183714 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:35.183721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:35.183728 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:35.183938 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:35.184464 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:35.184468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:35.184470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:35.185081 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:35.185319 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:35.185859 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 06:25:35.186297 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":507,"threads_started":1}
I20260809 06:25:35.190234 14621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220a7c74a9a248a4b9586ec65834e898" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } }
I20260809 06:25:35.190618 14621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:35.190730 14621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220a7c74a9a248a4b9586ec65834e898, State: Initialized, Role: FOLLOWER
I20260809 06:25:35.191326 14621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220a7c74a9a248a4b9586ec65834e898" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } }
I20260809 06:25:35.191571 14621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:35.191737 14621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:35.191936 14621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:35.194089 14621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220a7c74a9a248a4b9586ec65834e898" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } }
I20260809 06:25:35.194577 14621 leader_election.cc:302] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 220a7c74a9a248a4b9586ec65834e898; no voters:
I20260809 06:25:35.195188 14621 leader_election.cc:288] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:35.195353 14629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:35.196504 14629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [term 1 LEADER]: Becoming Leader. State: Replica: 220a7c74a9a248a4b9586ec65834e898, State: Running, Role: LEADER
I20260809 06:25:35.197183 14629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220a7c74a9a248a4b9586ec65834e898" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } }
I20260809 06:25:35.197541 14621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:35.200148 14632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 220a7c74a9a248a4b9586ec65834e898. Latest consensus state: current_term: 1 leader_uuid: "220a7c74a9a248a4b9586ec65834e898" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220a7c74a9a248a4b9586ec65834e898" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } } }
I20260809 06:25:35.200735 14632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:35.200559 14630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "220a7c74a9a248a4b9586ec65834e898" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220a7c74a9a248a4b9586ec65834e898" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } } }
I20260809 06:25:35.200960 14630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:35.206871 14642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:35.211397 14642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:35.219223 14642 catalog_manager.cc:1347] Generated new cluster ID: e0ed70f5db304a79a1cc7890cc477568
I20260809 06:25:35.219358 14642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:35.231942 14642 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:35.233862 14642 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:35.243479 14642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 220a7c74a9a248a4b9586ec65834e898: Generated new TSK 0
I20260809 06:25:35.244594 14642 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:25:35.299916 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45795
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:35.636545 14662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:35.636860 14662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:35.637024 14662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:35.644426 14662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:35.644550 14662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:35.644744 14662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:35.654695 14662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:35.656124 14662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:35.657482 14662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:35.670526 14669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:35.671011 14672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:35.670966 14668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:35.671723 14662 server_base.cc:1034] running on GCE node
I20260809 06:25:35.672472 14662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:35.673413 14662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:35.674614 14662 hybrid_clock.cc:648] HybridClock initialized: now 1786256735674574 us; error 38 us; skew 500 ppm
I20260809 06:25:35.674953 14662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:35 dist-test-slave-mjjr krb5kdc[14472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256735, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:35.678184 14662 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:25:35.679708 14662 webserver.cc:460] Webserver started at http://127.8.131.1:35599/ using document root <none> and password file <none>
I20260809 06:25:35.680397 14662 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:35.680564 14662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:35.680881 14662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:35.684103 14662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "ade7a9c4944044019060f9862bb5fa5a"
format_stamp: "Formatted at 2026-08-09 06:25:35 on dist-test-slave-mjjr"
server_key: "302aa099d2ac83ed18fd69d5ad3becec"
server_key_iv: "11f2e57d1cbaee163d4ebc8892800998"
server_key_version: "encryptionkey@0"
I20260809 06:25:35.684937 14662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "ade7a9c4944044019060f9862bb5fa5a"
format_stamp: "Formatted at 2026-08-09 06:25:35 on dist-test-slave-mjjr"
server_key: "302aa099d2ac83ed18fd69d5ad3becec"
server_key_iv: "11f2e57d1cbaee163d4ebc8892800998"
server_key_version: "encryptionkey@0"
I20260809 06:25:35.690085 14662 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:35.693568 14682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:35.694900 14662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:25:35.695077 14662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "ade7a9c4944044019060f9862bb5fa5a"
format_stamp: "Formatted at 2026-08-09 06:25:35 on dist-test-slave-mjjr"
server_key: "302aa099d2ac83ed18fd69d5ad3becec"
server_key_iv: "11f2e57d1cbaee163d4ebc8892800998"
server_key_version: "encryptionkey@0"
I20260809 06:25:35.695286 14662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:35.711469 14662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:35.715466 14662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:35.715838 14662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:35.716843 14662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:35.718119 14662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:35.718227 14662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:35.718386 14662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:35.718459 14662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:35.752086 14662 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:33209
I20260809 06:25:35.752137 14873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:33209 every 8 connection(s)
I20260809 06:25:35.753432 14662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:35.763969 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 14662
I20260809 06:25:35.764211 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:35.764966 8716 external_mini_cluster.cc:1428] Setting key 1a008ab3f886a9c732d743ff8711c6c6
Aug 09 06:25:35 dist-test-slave-mjjr krb5kdc[14472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256735, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:35.773972 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:35.756130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:40727 (local address 127.8.131.62:45795)
0809 06:25:35.756627 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:35.756632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:35.757627 (+ 995us) server_negotiation.cc:407] Connection header received
0809 06:25:35.759464 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:35.759469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:35.759563 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:35.759726 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:35.761244 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:35.762007 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:35.763578 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:35.764096 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:35.764250 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:35.768237 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:35.768242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:35.768246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:35.768278 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:35.770809 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:35.771558 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:35.771563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:35.771567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:35.771749 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:35.772207 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:35.772213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:35.772217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:35.772491 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:35.772643 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:35.773380 (+ 737us) server_negotiation.cc:299] Negotiation successful
0809 06:25:35.773617 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":267,"threads_started":1}
I20260809 06:25:35.774966 14882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:35.756727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45795 (local address 127.8.131.1:40727)
0809 06:25:35.757462 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:35.757499 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:35.759187 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:35.759962 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:35.759971 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:35.760485 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:35.761057 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:35.761073 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:35.762202 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:35.762207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:35.763417 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:35.763429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:35.764351 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:35.764361 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:35.764610 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:35.765385 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:35.765407 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:35.768010 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:35.770965 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:35.770974 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:35.770991 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:35.771421 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:35.771862 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:35.771868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:35.771871 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:35.772086 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:35.772633 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:35.772641 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:35.772998 (+ 357us) client_negotiation.cc:769] Sending connection context
0809 06:25:35.773271 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 06:25:35.773814 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":295,"threads_started":1}
I20260809 06:25:35.776603 14876 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45795
I20260809 06:25:35.776909 14876 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:35.777633 14876 heartbeater.cc:511] Master 127.8.131.62:45795 requested a full tablet report, sending...
I20260809 06:25:35.779978 14546 ts_manager.cc:194] Registered new tserver with Master: ade7a9c4944044019060f9862bb5fa5a (127.8.131.1:33209)
I20260809 06:25:35.782325 14546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:40727
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:25:35.815114 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45795
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:36.147382 14886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:36.147764 14886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:36.147888 14886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:36.156214 14886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:36.156325 14886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:36.156499 14886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:36.167455 14886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:36.168928 14886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:36.170308 14886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:36.185196 14895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:36.185140 14896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:36.185711 14899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:36.189180 14886 server_base.cc:1034] running on GCE node
I20260809 06:25:36.189872 14886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:36.190805 14886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:36.192013 14886 hybrid_clock.cc:648] HybridClock initialized: now 1786256736191949 us; error 52 us; skew 500 ppm
I20260809 06:25:36.192361 14886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:36 dist-test-slave-mjjr krb5kdc[14472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256736, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:36.195823 14886 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:25:36.197412 14886 webserver.cc:460] Webserver started at http://127.8.131.2:39347/ using document root <none> and password file <none>
I20260809 06:25:36.198180 14886 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:36.198324 14886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:36.198690 14886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:36.201798 14886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "a6e942e31a754d589b4aeab381212a8a"
format_stamp: "Formatted at 2026-08-09 06:25:36 on dist-test-slave-mjjr"
server_key: "ce6afa71d452fca07c85d49f77b43d24"
server_key_iv: "c7eb9b37cb51d943e482082a03517be7"
server_key_version: "encryptionkey@0"
I20260809 06:25:36.202596 14886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "a6e942e31a754d589b4aeab381212a8a"
format_stamp: "Formatted at 2026-08-09 06:25:36 on dist-test-slave-mjjr"
server_key: "ce6afa71d452fca07c85d49f77b43d24"
server_key_iv: "c7eb9b37cb51d943e482082a03517be7"
server_key_version: "encryptionkey@0"
I20260809 06:25:36.207942 14886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:25:36.211285 14907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:36.212558 14886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:36.212742 14886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "a6e942e31a754d589b4aeab381212a8a"
format_stamp: "Formatted at 2026-08-09 06:25:36 on dist-test-slave-mjjr"
server_key: "ce6afa71d452fca07c85d49f77b43d24"
server_key_iv: "c7eb9b37cb51d943e482082a03517be7"
server_key_version: "encryptionkey@0"
I20260809 06:25:36.212947 14886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:36.235296 14886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:36.239786 14886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:36.240113 14886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:36.241089 14886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:36.242437 14886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:36.242551 14886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:36.242705 14886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:36.242782 14886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:36.277316 14886 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:41785
I20260809 06:25:36.277432 15095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:41785 every 8 connection(s)
I20260809 06:25:36.278644 14886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:36.287859 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 14886
I20260809 06:25:36.288077 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:36.288584 8716 external_mini_cluster.cc:1428] Setting key e440d05bfe78d68a56affeb55d9e170e
Aug 09 06:25:36 dist-test-slave-mjjr krb5kdc[14472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256736, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:36.300123 15100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:36.281181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:51967 (local address 127.8.131.62:45795)
0809 06:25:36.281595 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:36.281600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:36.282593 (+ 993us) server_negotiation.cc:407] Connection header received
0809 06:25:36.284429 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:36.284434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:36.284503 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:36.284602 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:36.286101 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.286872 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.288561 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.289170 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.289290 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:36.292925 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:36.292933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:36.292941 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:36.292987 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:36.296649 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.297378 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.297386 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.297392 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.297572 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.298011 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.298019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.298025 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.298432 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:36.298591 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:36.299531 (+ 940us) server_negotiation.cc:299] Negotiation successful
0809 06:25:36.299779 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":207,"threads_started":1}
I20260809 06:25:36.301461 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:36.281742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45795 (local address 127.8.131.2:51967)
0809 06:25:36.282430 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:36.282467 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:36.284183 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:36.284791 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:36.284798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:36.285302 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:36.285924 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:36.285942 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.286997 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.287001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:36.288413 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:36.288425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.289412 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.289421 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:36.289638 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:36.290300 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:36.290321 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:36.292677 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:36.296778 (+ 4101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:36.296784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:36.296795 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:36.297256 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:36.297675 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:36.297679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:36.297681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:36.297867 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:36.298577 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:36.298584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:36.299110 (+ 526us) client_negotiation.cc:769] Sending connection context
0809 06:25:36.299430 (+ 320us) client_negotiation.cc:240] Negotiation successful
0809 06:25:36.300043 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":273,"threads_started":1}
I20260809 06:25:36.302947 15096 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45795
I20260809 06:25:36.303293 15096 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:36.304096 15096 heartbeater.cc:511] Master 127.8.131.62:45795 requested a full tablet report, sending...
I20260809 06:25:36.305797 14546 ts_manager.cc:194] Registered new tserver with Master: a6e942e31a754d589b4aeab381212a8a (127.8.131.2:41785)
I20260809 06:25:36.306641 14546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:51967
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:25:36.341077 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45795
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:36.674151 15106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:36.674463 15106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:36.674640 15106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:36.681802 15106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:36.681905 15106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:36.682015 15106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:36.691607 15106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:36.692917 15106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:36.694168 15106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:36.708514 15117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:36.708559 15119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:36.708634 15116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:36.709628 15106 server_base.cc:1034] running on GCE node
I20260809 06:25:36.710368 15106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:36.711423 15106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:36.712669 15106 hybrid_clock.cc:648] HybridClock initialized: now 1786256736712600 us; error 83 us; skew 500 ppm
I20260809 06:25:36.713043 15106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:36 dist-test-slave-mjjr krb5kdc[14472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256736, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:36.716364 15106 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:25:36.717868 15106 webserver.cc:460] Webserver started at http://127.8.131.3:45323/ using document root <none> and password file <none>
I20260809 06:25:36.718502 15106 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:36.718604 15106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:36.718860 15106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:36.721787 15106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "81e2124a56ce489b878069962f042976"
format_stamp: "Formatted at 2026-08-09 06:25:36 on dist-test-slave-mjjr"
server_key: "f97e7878e3c5e204205f1c3cf0a191e7"
server_key_iv: "91234d6986f42c080f6112e5e28d83ec"
server_key_version: "encryptionkey@0"
I20260809 06:25:36.722541 15106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "81e2124a56ce489b878069962f042976"
format_stamp: "Formatted at 2026-08-09 06:25:36 on dist-test-slave-mjjr"
server_key: "f97e7878e3c5e204205f1c3cf0a191e7"
server_key_iv: "91234d6986f42c080f6112e5e28d83ec"
server_key_version: "encryptionkey@0"
I20260809 06:25:36.727839 15106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:25:36.731135 15131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:36.732620 15106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:36.732758 15106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "81e2124a56ce489b878069962f042976"
format_stamp: "Formatted at 2026-08-09 06:25:36 on dist-test-slave-mjjr"
server_key: "f97e7878e3c5e204205f1c3cf0a191e7"
server_key_iv: "91234d6986f42c080f6112e5e28d83ec"
server_key_version: "encryptionkey@0"
I20260809 06:25:36.732933 15106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:36.763823 15106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:36.768397 15106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:36.768668 15106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:36.769623 15106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:36.770958 15106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:36.771031 15106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:36.771114 15106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:36.771158 15106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:36.785559 14876 heartbeater.cc:503] Master 127.8.131.62:45795 was elected leader, sending a full tablet report...
I20260809 06:25:36.806383 15106 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:44335
I20260809 06:25:36.806411 15331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:44335 every 8 connection(s)
I20260809 06:25:36.807928 15106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:36.811371 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 15106
I20260809 06:25:36.811582 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:36.812068 8716 external_mini_cluster.cc:1428] Setting key d3545252c9efc82e0a753616da8bbbcd
Aug 09 06:25:36 dist-test-slave-mjjr krb5kdc[14472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256736, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:36.828809 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:36.810648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:50235 (local address 127.8.131.62:45795)
0809 06:25:36.811161 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:36.811170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:36.812365 (+ 1195us) server_negotiation.cc:407] Connection header received
0809 06:25:36.814674 (+ 2309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:36.814679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:36.814792 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:36.814923 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:36.816596 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.817548 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.819095 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.819504 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.819641 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:36.823078 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:36.823084 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:36.823087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:36.823119 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:36.825545 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.826491 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.826495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.826497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.826615 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.827186 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.827191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.827193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.827553 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:36.827771 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:36.828392 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 06:25:36.828549 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":242,"threads_started":1}
I20260809 06:25:36.829752 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:36.811263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45795 (local address 127.8.131.3:50235)
0809 06:25:36.812171 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:36.812225 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:36.814388 (+ 2163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:36.815162 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:36.815170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:36.815775 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:36.816427 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:36.816453 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.817695 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.817701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:36.818943 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:36.818956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.819768 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.819777 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:36.820003 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:36.820604 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:36.820626 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:36.822861 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:36.825682 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:36.825690 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:36.825705 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:36.826359 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:36.826763 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:36.826766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:36.826769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:36.827057 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:36.827740 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:36.827746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:36.828050 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 06:25:36.828271 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 06:25:36.828638 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":446,"threads_started":1}
I20260809 06:25:36.831035 15332 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45795
I20260809 06:25:36.831293 15332 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:36.832011 15332 heartbeater.cc:511] Master 127.8.131.62:45795 requested a full tablet report, sending...
I20260809 06:25:36.833323 14546 ts_manager.cc:194] Registered new tserver with Master: 81e2124a56ce489b878069962f042976 (127.8.131.3:44335)
I20260809 06:25:36.834115 14546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:50235
I20260809 06:25:36.839754 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 06:25:36 dist-test-slave-mjjr krb5kdc[14472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 06:25:36 dist-test-slave-mjjr krb5kdc[14472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:36.891779 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:36.876916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.8.131.1:33209)
0809 06:25:36.877573 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:36.877581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:36.877608 (+ 27us) server_negotiation.cc:407] Connection header received
0809 06:25:36.877668 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:36.877674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:36.877896 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:36.878025 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:36.879367 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.880439 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.881586 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.882124 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.882219 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:36.886114 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:36.886125 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:36.886138 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:36.886168 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:36.889300 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.889905 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.889912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.889915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.890052 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.890424 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.890428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.890431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.890758 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:36.890989 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:36.891216 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 06:25:36.891417 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":330,"threads_started":1}
Aug 09 06:25:36 dist-test-slave-mjjr krb5kdc[14472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:36.909585 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:36.896170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.8.131.62:45795)
0809 06:25:36.896371 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:36.896376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:36.896399 (+ 23us) server_negotiation.cc:407] Connection header received
0809 06:25:36.896537 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:36.896541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:36.896617 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:36.896699 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:36.897936 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.898716 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.900102 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.900526 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.900621 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:36.904570 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:36.904575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:36.904578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:36.904613 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:36.907065 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.907632 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.907635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.907897 (+ 262us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.908014 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.908352 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.908357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.908361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.908676 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:36.908876 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:36.909142 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 06:25:36.909302 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 06:25:36.910332 14546 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60584
I20260809 06:25:36.911303 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 14662
I20260809 06:25:36.923738 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:36.912356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.8.131.1:33209)
0809 06:25:36.912570 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:36.912577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:36.912712 (+ 135us) server_negotiation.cc:407] Connection header received
0809 06:25:36.912826 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:36.912830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:36.912899 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:36.912978 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:36.914092 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.914751 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.915980 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:36.916483 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:36.916582 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:36.918822 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:36.918826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:36.918829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:36.918858 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:36.921521 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.922120 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.922124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.922126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.922241 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:36.922534 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:36.922537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:36.922539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:36.922762 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:36.922942 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:36.923194 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 06:25:36.923344 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:25:36.924422 14863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:37.008260 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 14662
I20260809 06:25:37.032120 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 14886
Aug 09 06:25:37 dist-test-slave-mjjr krb5kdc[14472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:25:37.048163 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:37.033154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58276 (local address 127.8.131.2:41785)
0809 06:25:37.033697 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:37.033704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:37.033729 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:25:37.033788 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:37.033795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:37.034025 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:37.034151 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:37.035456 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:37.036673 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:37.038037 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:37.038553 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:37.038654 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:37.042790 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:37.042801 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:37.042835 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:37.042876 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:37.045596 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:37.046207 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:37.046213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:37.046216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:37.046341 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:37.046679 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:37.046682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:37.046684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:37.047004 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:37.047234 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:37.047474 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 06:25:37.047758 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":290,"threads_started":1}
I20260809 06:25:37.049470 15090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:37.132308 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 14886
I20260809 06:25:37.154873 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 15106
Aug 09 06:25:37 dist-test-slave-mjjr krb5kdc[14472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:25:37.172355 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:37.155859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60406 (local address 127.8.131.3:44335)
0809 06:25:37.156568 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:37.156576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:37.156606 (+ 30us) server_negotiation.cc:407] Connection header received
0809 06:25:37.156667 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:37.156673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:37.156899 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:37.157032 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:37.158647 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:37.159928 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:37.161177 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:37.161724 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:37.161828 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:37.166261 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:37.166273 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:37.166288 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:37.166330 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:37.169288 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:37.170010 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:37.170019 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:37.170021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:37.170152 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:37.170575 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:37.170580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:37.170585 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:37.170949 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:37.171294 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:37.171610 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 06:25:37.172050 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":395,"threads_started":1}
I20260809 06:25:37.173664 15320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:37.257087 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 15106
I20260809 06:25:37.282793 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 14490
I20260809 06:25:37.284067 14608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:37.389514 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 14490
2026-08-09T06:25:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4205 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:37 dist-test-slave-mjjr krb5kdc[15371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:37 dist-test-slave-mjjr krb5kdc[15371](info): set up 2 sockets
Aug 09 06:25:37 dist-test-slave-mjjr krb5kdc[15371](info): commencing operation
krb5kdc: starting...
W20260809 06:25:38.678802 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:38 dist-test-slave-mjjr krb5kdc[15371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:38.894989 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:35047
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:35047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:39.228022 15400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:39.228377 15400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:39.228546 15400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:39.235934 15400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:39.236083 15400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:39.236197 15400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:39.236311 15400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:39.236402 15400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:39.246817 15400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:35047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:35047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:39.248281 15400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:39.249661 15400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:39.261960 15412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:39.262022 15409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:39.261988 15415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:39.263324 15400 server_base.cc:1034] running on GCE node
I20260809 06:25:39.264182 15400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:39.265564 15400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:39.266795 15400 hybrid_clock.cc:648] HybridClock initialized: now 1786256739266743 us; error 38 us; skew 500 ppm
I20260809 06:25:39.267225 15400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:39 dist-test-slave-mjjr krb5kdc[15371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256739, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:39.270519 15400 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:39.271790 15400 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:39.271936 15400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:39.272363 15400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:39.275534 15400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "15588f67318942d2b7647aca83825da3"
format_stamp: "Formatted at 2026-08-09 06:25:39 on dist-test-slave-mjjr"
server_key: "3b80f9e8d8000ad524e82f799708b3e0"
server_key_iv: "8be39a922d6274fc308d364263963047"
server_key_version: "encryptionkey@0"
I20260809 06:25:39.276494 15400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "15588f67318942d2b7647aca83825da3"
format_stamp: "Formatted at 2026-08-09 06:25:39 on dist-test-slave-mjjr"
server_key: "3b80f9e8d8000ad524e82f799708b3e0"
server_key_iv: "8be39a922d6274fc308d364263963047"
server_key_version: "encryptionkey@0"
I20260809 06:25:39.281915 15400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 06:25:39.285450 15426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:39.286767 15400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:25:39.286954 15400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "15588f67318942d2b7647aca83825da3"
format_stamp: "Formatted at 2026-08-09 06:25:39 on dist-test-slave-mjjr"
server_key: "3b80f9e8d8000ad524e82f799708b3e0"
server_key_iv: "8be39a922d6274fc308d364263963047"
server_key_version: "encryptionkey@0"
I20260809 06:25:39.287213 15400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:39.310954 15400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:39.314999 15400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:39.315317 15400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:39.334000 15400 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:35047
I20260809 06:25:39.334025 15510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:35047 every 8 connection(s)
I20260809 06:25:39.334936 15400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:39.336054 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 15400
I20260809 06:25:39.336333 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:39.337152 8716 external_mini_cluster.cc:1428] Setting key 11aad3c2f22a20ff0ec20553bd2299ca
I20260809 06:25:39.339890 15511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 06:25:39 dist-test-slave-mjjr krb5kdc[15371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:39.353181 15511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3: Bootstrap starting.
I20260809 06:25:39.356462 15511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:39.357839 15511 log.cc:824] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:39.361320 15511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3: No bootstrap required, opened a new log
I20260809 06:25:39.361550 15515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:39.339290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45850 (local address 127.8.131.62:35047)
0809 06:25:39.340143 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:39.340155 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:39.340198 (+ 43us) server_negotiation.cc:407] Connection header received
0809 06:25:39.341842 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:39.341862 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:39.342249 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:39.342572 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:39.344591 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:39.345967 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:39.347820 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:39.348476 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:39.348611 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:39.353353 (+ 4742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:39.353364 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:39.353387 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:39.353430 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:39.356551 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:39.357529 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:39.357542 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:39.357553 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:39.357881 (+ 328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:39.358356 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:39.358364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:39.358367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:39.359047 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:39.359346 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:39.359798 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 06:25:39.360233 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":413,"threads_started":1}
I20260809 06:25:39.369491 15511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15588f67318942d2b7647aca83825da3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35047 } }
I20260809 06:25:39.369828 15511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:39.369912 15511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15588f67318942d2b7647aca83825da3, State: Initialized, Role: FOLLOWER
I20260809 06:25:39.370379 15511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15588f67318942d2b7647aca83825da3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35047 } }
I20260809 06:25:39.370587 15511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:39.370682 15511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:39.370808 15511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:39.372957 15511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15588f67318942d2b7647aca83825da3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35047 } }
I20260809 06:25:39.373382 15511 leader_election.cc:302] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15588f67318942d2b7647aca83825da3; no voters:
I20260809 06:25:39.373874 15511 leader_election.cc:288] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:39.374039 15517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:39.374835 15517 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [term 1 LEADER]: Becoming Leader. State: Replica: 15588f67318942d2b7647aca83825da3, State: Running, Role: LEADER
I20260809 06:25:39.375413 15517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15588f67318942d2b7647aca83825da3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35047 } }
I20260809 06:25:39.376264 15511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:39.378453 15521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15588f67318942d2b7647aca83825da3. Latest consensus state: current_term: 1 leader_uuid: "15588f67318942d2b7647aca83825da3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15588f67318942d2b7647aca83825da3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35047 } } }
I20260809 06:25:39.378938 15521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:39.382477 15529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:39.382270 15520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15588f67318942d2b7647aca83825da3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15588f67318942d2b7647aca83825da3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35047 } } }
I20260809 06:25:39.382601 15520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:39.387004 15529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:39.396561 15529 catalog_manager.cc:1347] Generated new cluster ID: 356d19c8a8e9443cbeb3cf90bf076edf
I20260809 06:25:39.396683 15529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:39.409302 15529 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:39.411126 15529 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:39.423852 15529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3: Generated new TSK 0
I20260809 06:25:39.424898 15529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:25:39.481096 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:35047
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:39.812994 15550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:39.813258 15550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:39.813356 15550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:39.820586 15550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:39.820695 15550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:39.820812 15550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:39.830302 15550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:35047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:39.831475 15550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:39.833266 15550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:39.845849 15555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:39.845772 15560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:39.845816 15556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:39.847472 15550 server_base.cc:1034] running on GCE node
I20260809 06:25:39.848129 15550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:39.849020 15550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:39.850248 15550 hybrid_clock.cc:648] HybridClock initialized: now 1786256739850206 us; error 58 us; skew 500 ppm
I20260809 06:25:39.850546 15550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:39 dist-test-slave-mjjr krb5kdc[15371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256739, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:39.853684 15550 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:25:39.854799 15550 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:39.854898 15550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:39.855155 15550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:39.857976 15550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "827e7fe101fa4a1ba5ead583ce9298d0"
format_stamp: "Formatted at 2026-08-09 06:25:39 on dist-test-slave-mjjr"
server_key: "a3b7032fa809731bb3eb3b70ee57c32b"
server_key_iv: "b4c2e39a0ae204481c832f6bd31b862f"
server_key_version: "encryptionkey@0"
I20260809 06:25:39.858721 15550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "827e7fe101fa4a1ba5ead583ce9298d0"
format_stamp: "Formatted at 2026-08-09 06:25:39 on dist-test-slave-mjjr"
server_key: "a3b7032fa809731bb3eb3b70ee57c32b"
server_key_iv: "b4c2e39a0ae204481c832f6bd31b862f"
server_key_version: "encryptionkey@0"
I20260809 06:25:39.863781 15550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260809 06:25:39.867344 15569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:39.869196 15550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:39.869354 15550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "827e7fe101fa4a1ba5ead583ce9298d0"
format_stamp: "Formatted at 2026-08-09 06:25:39 on dist-test-slave-mjjr"
server_key: "a3b7032fa809731bb3eb3b70ee57c32b"
server_key_iv: "b4c2e39a0ae204481c832f6bd31b862f"
server_key_version: "encryptionkey@0"
I20260809 06:25:39.869606 15550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:39.884099 15550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:39.887852 15550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:39.888113 15550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:39.888999 15550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:39.890408 15550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:39.890496 15550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:39.890569 15550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:39.890673 15550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:39.924367 15550 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:35163
I20260809 06:25:39.924427 15743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:35163 every 8 connection(s)
I20260809 06:25:39.925269 15550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:39.933831 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 15550
I20260809 06:25:39.934075 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:39.934763 8716 external_mini_cluster.cc:1428] Setting key 899d29058223593199c1115ac47de901
Aug 09 06:25:39 dist-test-slave-mjjr krb5kdc[15371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256739, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:39.945731 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:39.927724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:59843 (local address 127.8.131.62:35047)
0809 06:25:39.928201 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:39.928205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:39.928990 (+ 785us) server_negotiation.cc:407] Connection header received
0809 06:25:39.930780 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:39.930786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:39.930872 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:39.931007 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:39.932555 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:39.933412 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:39.935207 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:39.935702 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:39.935837 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:39.940033 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:39.940038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:39.940041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:39.940073 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:39.942604 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:39.943358 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:39.943363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:39.943365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:39.943480 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:39.943965 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:39.943975 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:39.943978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:39.944230 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:39.944436 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:39.945129 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 06:25:39.945367 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":269,"threads_started":1}
I20260809 06:25:39.947022 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:39.928151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:35047 (local address 127.8.131.1:59843)
0809 06:25:39.928827 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:39.928870 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:39.930513 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:39.931231 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:39.931239 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:39.931825 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:39.932374 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:39.932401 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:39.933589 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:39.933599 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:39.935056 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:39.935068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:39.935956 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:39.935964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:39.936228 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:39.937038 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:39.937068 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:39.939757 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:39.942758 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:39.942766 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:39.942780 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:39.943205 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:39.943600 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:39.943605 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:39.943609 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:39.943833 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:39.944409 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:39.944417 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:39.944736 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 06:25:39.945011 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 06:25:39.945535 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":289,"threads_started":1}
I20260809 06:25:39.948776 15745 heartbeater.cc:347] Connected to a master server at 127.8.131.62:35047
I20260809 06:25:39.949106 15745 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:39.949937 15745 heartbeater.cc:511] Master 127.8.131.62:35047 requested a full tablet report, sending...
I20260809 06:25:39.952387 15456 ts_manager.cc:194] Registered new tserver with Master: 827e7fe101fa4a1ba5ead583ce9298d0 (127.8.131.1:35163)
I20260809 06:25:39.954599 15456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:59843
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:25:39.987444 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:35047
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:40.314845 15755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:40.315111 15755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:40.315275 15755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:40.322419 15755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:40.322520 15755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:40.322731 15755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:40.332427 15755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:35047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:40.333966 15755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:40.335616 15755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:40.350556 15763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:40.350478 15764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:40.351076 15766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:40.351541 15755 server_base.cc:1034] running on GCE node
I20260809 06:25:40.352231 15755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:40.353252 15755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:40.354492 15755 hybrid_clock.cc:648] HybridClock initialized: now 1786256740354438 us; error 46 us; skew 500 ppm
I20260809 06:25:40.354816 15755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:40 dist-test-slave-mjjr krb5kdc[15371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256740, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:40.362308 15755 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:25:40.363479 15755 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:40.363626 15755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:40.364008 15755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:40.367026 15755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "0b0af0c319c94c988f020f72600936f8"
format_stamp: "Formatted at 2026-08-09 06:25:40 on dist-test-slave-mjjr"
server_key: "5fb7be79aaa538c83c7f99710c64e556"
server_key_iv: "c1d030b4968842d16eff157e1e522b04"
server_key_version: "encryptionkey@0"
I20260809 06:25:40.367946 15755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "0b0af0c319c94c988f020f72600936f8"
format_stamp: "Formatted at 2026-08-09 06:25:40 on dist-test-slave-mjjr"
server_key: "5fb7be79aaa538c83c7f99710c64e556"
server_key_iv: "c1d030b4968842d16eff157e1e522b04"
server_key_version: "encryptionkey@0"
I20260809 06:25:40.373123 15755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 06:25:40.376875 15773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:40.378247 15755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 06:25:40.378525 15755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "0b0af0c319c94c988f020f72600936f8"
format_stamp: "Formatted at 2026-08-09 06:25:40 on dist-test-slave-mjjr"
server_key: "5fb7be79aaa538c83c7f99710c64e556"
server_key_iv: "c1d030b4968842d16eff157e1e522b04"
server_key_version: "encryptionkey@0"
I20260809 06:25:40.378830 15755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:40.396569 15755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:40.400630 15755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:40.400913 15755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:40.401901 15755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:40.403227 15755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:40.403352 15755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:40.403489 15755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:40.403584 15755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:40.437795 15755 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:42827
I20260809 06:25:40.437851 15952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:42827 every 8 connection(s)
I20260809 06:25:40.438725 15755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:40.439980 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 15755
I20260809 06:25:40.440209 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:40.440711 8716 external_mini_cluster.cc:1428] Setting key 759d9453808f12e21655b35b264ecf7c
Aug 09 06:25:40 dist-test-slave-mjjr krb5kdc[15371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256740, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:40.463966 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:40.441525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:59413 (local address 127.8.131.62:35047)
0809 06:25:40.441923 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:40.441927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:40.443185 (+ 1258us) server_negotiation.cc:407] Connection header received
0809 06:25:40.446162 (+ 2977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:40.446167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:40.446234 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:40.446323 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:40.448543 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.449242 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.451789 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.452584 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.452733 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:40.456487 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:40.456496 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:40.456504 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:40.456550 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:40.460388 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:40.461200 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:40.461208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:40.461215 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:40.461379 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:40.461805 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:40.461813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:40.461819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:40.462249 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:40.462415 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:40.463167 (+ 752us) server_negotiation.cc:299] Negotiation successful
0809 06:25:40.463446 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286}
I20260809 06:25:40.464629 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:40.442112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:35047 (local address 127.8.131.2:59413)
0809 06:25:40.442994 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:40.443067 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:40.445857 (+ 2790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:40.446613 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:40.446626 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:40.447436 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:40.448341 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:40.448369 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.449403 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.449411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:40.451592 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:40.451611 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.452801 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.452805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:40.453019 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:40.453725 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:40.453748 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:40.456231 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:40.460557 (+ 4326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:40.460564 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:40.460577 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:40.461049 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:40.461494 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:40.461498 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:40.461500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:40.461670 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:40.462396 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:40.462402 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:40.462733 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 06:25:40.462983 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 06:25:40.463395 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":339,"threads_started":1}
I20260809 06:25:40.466311 15953 heartbeater.cc:347] Connected to a master server at 127.8.131.62:35047
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
I20260809 06:25:40.466673 15953 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:40.467345 15953 heartbeater.cc:511] Master 127.8.131.62:35047 requested a full tablet report, sending...
I20260809 06:25:40.468930 15456 ts_manager.cc:194] Registered new tserver with Master: 0b0af0c319c94c988f020f72600936f8 (127.8.131.2:42827)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
I20260809 06:25:40.469889 15456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:59413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:25:40.498354 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:35047
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:40.829334 15963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:40.829694 15963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:40.829888 15963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:40.837273 15963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:40.837433 15963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:40.837628 15963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:40.847687 15963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:35047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:40.849054 15963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:40.850360 15963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:40.863385 15974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:40.863428 15972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:40.863453 15971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:40.864171 15963 server_base.cc:1034] running on GCE node
I20260809 06:25:40.864795 15963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:40.865737 15963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:40.866938 15963 hybrid_clock.cc:648] HybridClock initialized: now 1786256740866895 us; error 33 us; skew 500 ppm
I20260809 06:25:40.867259 15963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:40 dist-test-slave-mjjr krb5kdc[15371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256740, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:40.870545 15963 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:25:40.871783 15963 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:40.871927 15963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:40.872264 15963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:40.875373 15963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "ff507c8e46b74151ab16471208486bd3"
format_stamp: "Formatted at 2026-08-09 06:25:40 on dist-test-slave-mjjr"
server_key: "3dbb51517f21feffb0bb7cb81107fcc1"
server_key_iv: "e83fdb717eddba63af65c6167908d51c"
server_key_version: "encryptionkey@0"
I20260809 06:25:40.876260 15963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "ff507c8e46b74151ab16471208486bd3"
format_stamp: "Formatted at 2026-08-09 06:25:40 on dist-test-slave-mjjr"
server_key: "3dbb51517f21feffb0bb7cb81107fcc1"
server_key_iv: "e83fdb717eddba63af65c6167908d51c"
server_key_version: "encryptionkey@0"
I20260809 06:25:40.881273 15963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:40.884892 15986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:40.886165 15963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:40.886355 15963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "ff507c8e46b74151ab16471208486bd3"
format_stamp: "Formatted at 2026-08-09 06:25:40 on dist-test-slave-mjjr"
server_key: "3dbb51517f21feffb0bb7cb81107fcc1"
server_key_iv: "e83fdb717eddba63af65c6167908d51c"
server_key_version: "encryptionkey@0"
I20260809 06:25:40.886617 15963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:40.902551 15963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:40.906417 15963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:40.906765 15963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:40.907768 15963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:40.909106 15963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:40.909195 15963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:40.909361 15963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:40.909459 15963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:40.944533 15963 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:33791
I20260809 06:25:40.944591 16147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:33791 every 8 connection(s)
I20260809 06:25:40.945516 15963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:40.949733 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 15963
I20260809 06:25:40.949963 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:40.950496 8716 external_mini_cluster.cc:1428] Setting key 17917b7b550bd4d59a9156923b2dd6eb
I20260809 06:25:40.958474 15745 heartbeater.cc:503] Master 127.8.131.62:35047 was elected leader, sending a full tablet report...
Aug 09 06:25:40 dist-test-slave-mjjr krb5kdc[15371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256740, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:40.966159 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:40.947967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:51841 (local address 127.8.131.62:35047)
0809 06:25:40.948152 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:40.948156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:40.949332 (+ 1176us) server_negotiation.cc:407] Connection header received
0809 06:25:40.951776 (+ 2444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:40.951780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:40.951950 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:40.952079 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:40.953803 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.954555 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.956180 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.956589 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.956730 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:40.960499 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:40.960504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:40.960507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:40.960536 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:40.962931 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:40.963608 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:40.963612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:40.963614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:40.963797 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:40.964252 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:40.964258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:40.964261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:40.964646 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:40.964830 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:40.965678 (+ 848us) server_negotiation.cc:299] Negotiation successful
0809 06:25:40.965913 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:25:40.967183 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:40.948488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:35047 (local address 127.8.131.3:51841)
0809 06:25:40.949172 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:40.949209 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:40.951423 (+ 2214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:40.952302 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:40.952311 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:40.952965 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:40.953623 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:40.953647 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.954708 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.954715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:40.956030 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:40.956042 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.956836 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.956845 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:40.957078 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:40.957720 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:40.957742 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:40.960319 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:40.963053 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:40.963059 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:40.963070 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:40.963499 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:40.963938 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:40.963943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:40.963946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:40.964124 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:40.964792 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:40.964799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:40.965170 (+ 371us) client_negotiation.cc:769] Sending connection context
0809 06:25:40.965509 (+ 339us) client_negotiation.cc:240] Negotiation successful
0809 06:25:40.966060 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":289,"threads_started":1}
I20260809 06:25:40.968575 16151 heartbeater.cc:347] Connected to a master server at 127.8.131.62:35047
I20260809 06:25:40.968888 16151 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:40.969537 16151 heartbeater.cc:511] Master 127.8.131.62:35047 requested a full tablet report, sending...
I20260809 06:25:40.970918 15456 ts_manager.cc:194] Registered new tserver with Master: ff507c8e46b74151ab16471208486bd3 (127.8.131.3:33791)
I20260809 06:25:40.971733 15456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:51841
I20260809 06:25:40.977982 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:25:40.994674 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:40.982709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.8.131.62:35047)
0809 06:25:40.982931 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:40.982940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:40.983533 (+ 593us) server_negotiation.cc:407] Connection header received
0809 06:25:40.983715 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:40.983719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:40.983777 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:40.983876 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:40.985296 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.985988 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.987175 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:40.987579 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:40.987754 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:40.989783 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:40.989787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:40.989790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:40.989840 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:40.992464 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:40.993077 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:40.993080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:40.993083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:40.993203 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:40.993527 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:40.993531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:40.993533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:40.993826 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:40.993997 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:40.994293 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 06:25:40.994445 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:25:40.998639 15456 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:25:41.005355 15456 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:41.029788 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.017492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:33791 (local address 127.0.0.1:47262)
0809 06:25:41.018428 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:41.018468 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:41.018634 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:41.022254 (+ 3620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:41.022260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:41.022298 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:41.022963 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.022975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.024986 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.024995 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.026945 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.026958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.028747 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.028754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.028944 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.029004 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 06:25:41.029147 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 06:25:41.029332 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":71,"thread_start_us":1330,"threads_started":1}
I20260809 06:25:41.029934 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.018135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35163 (local address 127.0.0.1:37414)
0809 06:25:41.021362 (+ 3227us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:41.021380 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:41.021577 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:41.022119 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:41.022125 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:41.022151 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:41.022738 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.022766 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.024859 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.024863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.027292 (+ 2429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.027305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.029132 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.029138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.029289 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.029319 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 06:25:41.029405 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 06:25:41.029534 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3063,"mutex_wait_us":840,"thread_start_us":1768,"threads_started":1}
I20260809 06:25:41.030503 16171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.017705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47262 (local address 127.8.131.3:33791)
0809 06:25:41.021667 (+ 3962us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:41.021675 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:41.021707 (+ 32us) server_negotiation.cc:407] Connection header received
0809 06:25:41.021812 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:41.021818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:41.022050 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:41.022537 (+ 487us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:41.023202 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.024791 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.027142 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.028550 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.028697 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.028951 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:41.030048 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0809 06:25:41.030221 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3782,"thread_start_us":3616,"threads_started":1}
I20260809 06:25:41.030553 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.018590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.8.131.1:35163)
0809 06:25:41.020371 (+ 1781us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:41.020394 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:41.021471 (+ 1077us) server_negotiation.cc:407] Connection header received
0809 06:25:41.021769 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:41.021774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:41.021937 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:41.022131 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:41.022967 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.024697 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.027480 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.028959 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.029078 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.029289 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:41.029463 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 06:25:41.029640 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1614,"thread_start_us":1483,"threads_started":1}
I20260809 06:25:41.030572 16172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.018328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58696 (local address 127.8.131.2:42827)
0809 06:25:41.019913 (+ 1585us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:41.019925 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:41.020972 (+ 1047us) server_negotiation.cc:407] Connection header received
0809 06:25:41.021320 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:41.021331 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:41.021595 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:41.021776 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:41.023157 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.024454 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.027786 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.029164 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.029264 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.029438 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:41.030085 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 06:25:41.030265 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1397,"thread_start_us":1534,"threads_started":1}
I20260809 06:25:41.031270 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.017626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:42827 (local address 127.0.0.1:58696)
0809 06:25:41.020838 (+ 3212us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:41.020862 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:41.021102 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:41.022095 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:41.022102 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:41.022133 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:41.022925 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.022948 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.024586 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.024594 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.027582 (+ 2988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.027597 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.029647 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.029654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.029861 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.029914 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 06:25:41.030752 (+ 838us) client_negotiation.cc:240] Negotiation successful
0809 06:25:41.030901 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3064,"mutex_wait_us":132,"thread_start_us":3343,"threads_started":1}
I20260809 06:25:41.034067 15650 tablet_service.cc:1467] Processing CreateTablet for tablet c975befb5b564ffd93322677ac5f1e90 (DEFAULT_TABLE table=test-table [id=553fdddee6a844ac8dc9c7dd2d4f4a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:41.034278 16055 tablet_service.cc:1467] Processing CreateTablet for tablet c975befb5b564ffd93322677ac5f1e90 (DEFAULT_TABLE table=test-table [id=553fdddee6a844ac8dc9c7dd2d4f4a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:41.034459 15847 tablet_service.cc:1467] Processing CreateTablet for tablet c975befb5b564ffd93322677ac5f1e90 (DEFAULT_TABLE table=test-table [id=553fdddee6a844ac8dc9c7dd2d4f4a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:41.035435 15650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c975befb5b564ffd93322677ac5f1e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:41.035624 16055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c975befb5b564ffd93322677ac5f1e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:41.035622 15847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c975befb5b564ffd93322677ac5f1e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:41.046842 16176 tablet_bootstrap.cc:492] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: Bootstrap starting.
I20260809 06:25:41.050019 16177 tablet_bootstrap.cc:492] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: Bootstrap starting.
I20260809 06:25:41.051597 16176 tablet_bootstrap.cc:654] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:41.052953 16176 log.cc:824] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:41.053699 16177 tablet_bootstrap.cc:654] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:41.053928 16178 tablet_bootstrap.cc:492] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: Bootstrap starting.
I20260809 06:25:41.055240 16177 log.cc:824] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:41.057070 16178 tablet_bootstrap.cc:654] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:41.058008 16178 log.cc:824] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:41.058316 16176 tablet_bootstrap.cc:492] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: No bootstrap required, opened a new log
I20260809 06:25:41.058641 16176 ts_tablet_manager.cc:1397] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.006s
I20260809 06:25:41.061163 16178 tablet_bootstrap.cc:492] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: No bootstrap required, opened a new log
I20260809 06:25:41.061164 16177 tablet_bootstrap.cc:492] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: No bootstrap required, opened a new log
I20260809 06:25:41.061445 16178 ts_tablet_manager.cc:1397] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 06:25:41.061445 16177 ts_tablet_manager.cc:1397] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.003s
I20260809 06:25:41.066084 16176 raft_consensus.cc:348] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } }
I20260809 06:25:41.066514 16176 raft_consensus.cc:374] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:41.066641 16176 raft_consensus.cc:729] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 827e7fe101fa4a1ba5ead583ce9298d0, State: Initialized, Role: FOLLOWER
I20260809 06:25:41.067298 16176 consensus_queue.cc:260] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } }
I20260809 06:25:41.069118 16176 ts_tablet_manager.cc:1428] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260809 06:25:41.072283 16178 raft_consensus.cc:348] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } }
I20260809 06:25:41.072739 16178 raft_consensus.cc:374] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:41.072528 16177 raft_consensus.cc:348] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } }
I20260809 06:25:41.072851 16178 raft_consensus.cc:729] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0af0c319c94c988f020f72600936f8, State: Initialized, Role: FOLLOWER
I20260809 06:25:41.072950 16177 raft_consensus.cc:374] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:41.073055 16177 raft_consensus.cc:729] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff507c8e46b74151ab16471208486bd3, State: Initialized, Role: FOLLOWER
I20260809 06:25:41.073493 16178 consensus_queue.cc:260] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } }
I20260809 06:25:41.073664 16177 consensus_queue.cc:260] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } }
I20260809 06:25:41.074719 15953 heartbeater.cc:503] Master 127.8.131.62:35047 was elected leader, sending a full tablet report...
I20260809 06:25:41.075393 16151 heartbeater.cc:503] Master 127.8.131.62:35047 was elected leader, sending a full tablet report...
I20260809 06:25:41.075951 16177 ts_tablet_manager.cc:1428] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260809 06:25:41.076327 16178 ts_tablet_manager.cc:1428] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260809 06:25:41.152966 16187 raft_consensus.cc:482] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:41.153299 16187 raft_consensus.cc:504] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } }
I20260809 06:25:41.155094 16187 leader_election.cc:288] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff507c8e46b74151ab16471208486bd3 (127.8.131.3:33791), 0b0af0c319c94c988f020f72600936f8 (127.8.131.2:42827)
I20260809 06:25:41.162338 16172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.155751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:36707 (local address 127.8.131.2:42827)
0809 06:25:41.156247 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:41.156254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:41.156811 (+ 557us) server_negotiation.cc:407] Connection header received
0809 06:25:41.157008 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:41.157014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:41.157094 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:41.157197 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:41.157829 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.158531 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.160527 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.161306 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.161473 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.161578 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:41.161955 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 06:25:41.162079 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336}
I20260809 06:25:41.162338 16171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.155557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:45639 (local address 127.8.131.3:33791)
0809 06:25:41.156063 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:41.156069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:41.156561 (+ 492us) server_negotiation.cc:407] Connection header received
0809 06:25:41.156784 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:41.156791 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:41.156883 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:41.156986 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:41.157667 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.158482 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.160471 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.161280 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.161380 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.161531 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:41.161928 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 06:25:41.162067 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322}
I20260809 06:25:41.162519 16197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.155318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:33791 (local address 127.8.131.1:45639)
0809 06:25:41.156398 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:41.156425 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:41.156617 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:41.157011 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:41.157016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:41.157054 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:41.157499 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.157511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.158622 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.158631 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.160305 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.160318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.161543 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.161551 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.161720 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.161780 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 06:25:41.161853 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 06:25:41.162016 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":56,"thread_start_us":994,"threads_started":1}
I20260809 06:25:41.162568 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.155435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:42827 (local address 127.8.131.1:36707)
0809 06:25:41.156689 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:41.156708 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:41.156841 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:41.157232 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:41.157239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:41.157259 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:41.157668 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.157680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.158707 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.158711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.160334 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:41.160346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.161596 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.161602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:41.161759 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.161788 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 06:25:41.161907 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 06:25:41.162032 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1111,"mutex_wait_us":27,"thread_start_us":512,"threads_started":1}
I20260809 06:25:41.163731 16088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c975befb5b564ffd93322677ac5f1e90" candidate_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff507c8e46b74151ab16471208486bd3" is_pre_election: true
I20260809 06:25:41.163878 15877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c975befb5b564ffd93322677ac5f1e90" candidate_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0af0c319c94c988f020f72600936f8" is_pre_election: true
I20260809 06:25:41.164403 16088 raft_consensus.cc:2454] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 827e7fe101fa4a1ba5ead583ce9298d0 in term 0.
I20260809 06:25:41.164392 15877 raft_consensus.cc:2454] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 827e7fe101fa4a1ba5ead583ce9298d0 in term 0.
I20260809 06:25:41.165247 15574 leader_election.cc:302] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0af0c319c94c988f020f72600936f8, 827e7fe101fa4a1ba5ead583ce9298d0; no voters:
I20260809 06:25:41.165685 16187 raft_consensus.cc:2781] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:41.165851 16187 raft_consensus.cc:482] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:41.165966 16187 raft_consensus.cc:3037] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:41.168253 16187 raft_consensus.cc:504] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } }
I20260809 06:25:41.168838 16187 leader_election.cc:288] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [CANDIDATE]: Term 1 election: Requested vote from peers ff507c8e46b74151ab16471208486bd3 (127.8.131.3:33791), 0b0af0c319c94c988f020f72600936f8 (127.8.131.2:42827)
I20260809 06:25:41.169349 16088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c975befb5b564ffd93322677ac5f1e90" candidate_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff507c8e46b74151ab16471208486bd3"
I20260809 06:25:41.169375 15877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c975befb5b564ffd93322677ac5f1e90" candidate_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0af0c319c94c988f020f72600936f8"
I20260809 06:25:41.169575 16088 raft_consensus.cc:3037] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:41.169584 15877 raft_consensus.cc:3037] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:41.171962 15877 raft_consensus.cc:2454] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 827e7fe101fa4a1ba5ead583ce9298d0 in term 1.
I20260809 06:25:41.172051 16088 raft_consensus.cc:2454] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 827e7fe101fa4a1ba5ead583ce9298d0 in term 1.
I20260809 06:25:41.172448 15574 leader_election.cc:302] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0af0c319c94c988f020f72600936f8, 827e7fe101fa4a1ba5ead583ce9298d0; no voters:
I20260809 06:25:41.172806 16187 raft_consensus.cc:2781] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:41.173568 16187 raft_consensus.cc:686] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 1 LEADER]: Becoming Leader. State: Replica: 827e7fe101fa4a1ba5ead583ce9298d0, State: Running, Role: LEADER
I20260809 06:25:41.174140 16187 consensus_queue.cc:237] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } }
W20260809 06:25:41.177425 15747 tablet.cc:2369] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:25:41.178835 15456 catalog_manager.cc:5495] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 827e7fe101fa4a1ba5ead583ce9298d0 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "827e7fe101fa4a1ba5ead583ce9298d0" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35163 } health_report { overall_health: HEALTHY } } }
W20260809 06:25:41.190766 15954 tablet.cc:2369] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:25:41.196964 16152 tablet.cc:2369] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:25:41.222668 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.217182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37416 (local address 127.8.131.1:35163)
0809 06:25:41.217417 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:41.217426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:41.217486 (+ 60us) server_negotiation.cc:407] Connection header received
0809 06:25:41.217688 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:41.217692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:41.217771 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:41.217921 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:25:41.218266 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.219213 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.220597 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.221029 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.221130 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.221585 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:25:41.222095 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:25:41.222204 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:41.222298 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 06:25:41.222365 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 06:25:41.231033 15877 raft_consensus.cc:1261] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [term 1 FOLLOWER]: Refusing update from remote peer 827e7fe101fa4a1ba5ead583ce9298d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:41.231033 16088 raft_consensus.cc:1261] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [term 1 FOLLOWER]: Refusing update from remote peer 827e7fe101fa4a1ba5ead583ce9298d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:41.232234 16198 consensus_queue.cc:1035] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff507c8e46b74151ab16471208486bd3" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:41.232611 16187 consensus_queue.cc:1035] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0af0c319c94c988f020f72600936f8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 42827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:41.248486 16206 mvcc.cc:205] Tried to move back new op lower bound from 7316507612071084032 to 7316507611853062144. Current Snapshot: MvccSnapshot[applied={T|T < 7316507612071084032}]
I20260809 06:25:41.258433 15455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:41.258692 15455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:41.262629 15455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15588f67318942d2b7647aca83825da3: Sending DeleteTablet for 3 replicas of tablet c975befb5b564ffd93322677ac5f1e90
I20260809 06:25:41.263880 15847 tablet_service.cc:1514] Processing DeleteTablet for tablet c975befb5b564ffd93322677ac5f1e90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:41 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:58696
I20260809 06:25:41.263880 16055 tablet_service.cc:1514] Processing DeleteTablet for tablet c975befb5b564ffd93322677ac5f1e90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:41 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:47262
I20260809 06:25:41.263952 15650 tablet_service.cc:1514] Processing DeleteTablet for tablet c975befb5b564ffd93322677ac5f1e90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:41 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:37414
I20260809 06:25:41.266597 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 15550
I20260809 06:25:41.280313 16223 tablet_replica.cc:330] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: stopping tablet replica
I20260809 06:25:41.281711 16223 raft_consensus.cc:2229] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:25:41.283731 16223 raft_consensus.cc:2258] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:41.285457 16222 tablet_replica.cc:330] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: stopping tablet replica
I20260809 06:25:41.285893 16222 raft_consensus.cc:2229] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:41.286430 16222 raft_consensus.cc:2258] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:41.287273 16218 tablet_replica.cc:330] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: stopping tablet replica
I20260809 06:25:41.287711 16218 raft_consensus.cc:2229] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:41.287890 16223 ts_tablet_manager.cc:1905] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:41.288177 16218 raft_consensus.cc:2258] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:41.290434 16218 ts_tablet_manager.cc:1905] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 06:25:41 dist-test-slave-mjjr krb5kdc[15371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:41.295513 16222 ts_tablet_manager.cc:1905] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:41.295732 16223 ts_tablet_manager.cc:1918] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:25:41.295907 16223 log.cc:1196] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/c975befb5b564ffd93322677ac5f1e90
I20260809 06:25:41.296484 16223 ts_tablet_manager.cc:1939] T c975befb5b564ffd93322677ac5f1e90 P 827e7fe101fa4a1ba5ead583ce9298d0: Deleting consensus metadata
I20260809 06:25:41.298321 15433 catalog_manager.cc:4841] TS 827e7fe101fa4a1ba5ead583ce9298d0 (127.8.131.1:35163): tablet c975befb5b564ffd93322677ac5f1e90 (table test-table [id=553fdddee6a844ac8dc9c7dd2d4f4a57]) successfully deleted
I20260809 06:25:41.298424 16218 ts_tablet_manager.cc:1918] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:25:41.298723 16218 log.cc:1196] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/c975befb5b564ffd93322677ac5f1e90
I20260809 06:25:41.299361 16218 ts_tablet_manager.cc:1939] T c975befb5b564ffd93322677ac5f1e90 P ff507c8e46b74151ab16471208486bd3: Deleting consensus metadata
I20260809 06:25:41.301160 15428 catalog_manager.cc:4841] TS ff507c8e46b74151ab16471208486bd3 (127.8.131.3:33791): tablet c975befb5b564ffd93322677ac5f1e90 (table test-table [id=553fdddee6a844ac8dc9c7dd2d4f4a57]) successfully deleted
I20260809 06:25:41.302484 16222 ts_tablet_manager.cc:1918] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:25:41.302660 16222 log.cc:1196] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/c975befb5b564ffd93322677ac5f1e90
I20260809 06:25:41.303220 16222 ts_tablet_manager.cc:1939] T c975befb5b564ffd93322677ac5f1e90 P 0b0af0c319c94c988f020f72600936f8: Deleting consensus metadata
I20260809 06:25:41.304850 15431 catalog_manager.cc:4841] TS 0b0af0c319c94c988f020f72600936f8 (127.8.131.2:42827): tablet c975befb5b564ffd93322677ac5f1e90 (table test-table [id=553fdddee6a844ac8dc9c7dd2d4f4a57]) successfully deleted
I20260809 06:25:41.306607 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.268379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.8.131.1:35163)
0809 06:25:41.268607 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:41.268616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:41.272604 (+ 3988us) server_negotiation.cc:407] Connection header received
0809 06:25:41.272803 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:41.272810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:41.272907 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:41.273045 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:41.281697 (+ 8652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.283000 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.288594 (+ 5594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.289330 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.289452 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.300000 (+ 10548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:41.300014 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:41.300030 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:41.300071 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:41.303596 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:41.304337 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:41.304343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:41.304346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:41.304487 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:41.304968 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:41.304976 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:41.304980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:41.305362 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:41.305668 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:41.305989 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 06:25:41.306241 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 06:25:41.307384 15739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:41.407094 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 15550
I20260809 06:25:41.429636 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 15755
Aug 09 06:25:41 dist-test-slave-mjjr krb5kdc[15371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:25:41.445511 16172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.430460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.8.131.2:42827)
0809 06:25:41.430656 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:41.430663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:41.430683 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:25:41.430892 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:41.430900 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:41.430990 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:41.431104 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:41.432439 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.433223 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.434474 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.434855 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.435012 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.439331 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:41.439344 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:41.439356 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:41.439395 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:41.442491 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:41.443173 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:41.443181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:41.443184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:41.443324 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:41.443892 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:41.443900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:41.443905 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:41.444295 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:41.444613 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:41.444929 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 06:25:41.445176 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 06:25:41.446280 15947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:41.539214 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 15755
I20260809 06:25:41.562394 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 15963
Aug 09 06:25:41 dist-test-slave-mjjr krb5kdc[15371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:25:41.577857 16171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:41.563277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47270 (local address 127.8.131.3:33791)
0809 06:25:41.563512 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:41.563518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:41.563539 (+ 21us) server_negotiation.cc:407] Connection header received
0809 06:25:41.563736 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:41.563742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:41.563816 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:41.563954 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:41.565230 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.565983 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.567333 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:41.567813 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:41.567967 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:41.572274 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:41.572290 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:41.572306 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:41.572341 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:41.574904 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:41.575627 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:41.575633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:41.575636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:41.576000 (+ 364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:41.576447 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:41.576450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:41.576452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:41.576833 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:41.577060 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:41.577388 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 06:25:41.577554 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 06:25:41.578555 16143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:41.673552 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 15963
I20260809 06:25:41.700757 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 15400
I20260809 06:25:41.702105 15504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:41.812206 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 15400
2026-08-09T06:25:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4424 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T06:25:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:41Z Disabled control of system clock
I20260809 06:25:41.894750 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:37549
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:37549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 06:25:42.219928 16233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:42.220222 16233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:42.220387 16233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:42.227663 16233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:42.227766 16233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:42.227837 16233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:42.227921 16233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:42.228004 16233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:42.237154 16233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:37549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:37549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:42.238559 16233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:42.240375 16233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:42.252310 16242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:42.252466 16243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:42.252341 16247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:42.253502 16233 server_base.cc:1034] running on GCE node
I20260809 06:25:42.254313 16233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:42.255774 16233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:42.257043 16233 hybrid_clock.cc:648] HybridClock initialized: now 1786256742256986 us; error 42 us; skew 500 ppm
I20260809 06:25:42.257488 16233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:25:42.259939 16233 webserver.cc:460] Webserver started at http://127.8.131.62:34339/ using document root <none> and password file <none>
I20260809 06:25:42.260668 16233 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:42.260828 16233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:42.261219 16233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:42.264462 16233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "98d687b7d9b643d2a13845dbb458d466"
format_stamp: "Formatted at 2026-08-09 06:25:42 on dist-test-slave-mjjr"
server_key: "b10e3e38cbc78c89843188ca768e6356"
server_key_iv: "c95a57e5fd1fd9b55fe5370fc1bb746f"
server_key_version: "encryptionkey@0"
I20260809 06:25:42.265378 16233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "98d687b7d9b643d2a13845dbb458d466"
format_stamp: "Formatted at 2026-08-09 06:25:42 on dist-test-slave-mjjr"
server_key: "b10e3e38cbc78c89843188ca768e6356"
server_key_iv: "c95a57e5fd1fd9b55fe5370fc1bb746f"
server_key_version: "encryptionkey@0"
I20260809 06:25:42.271047 16233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 06:25:42.274792 16257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:42.276409 16233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 06:25:42.276621 16233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "98d687b7d9b643d2a13845dbb458d466"
format_stamp: "Formatted at 2026-08-09 06:25:42 on dist-test-slave-mjjr"
server_key: "b10e3e38cbc78c89843188ca768e6356"
server_key_iv: "c95a57e5fd1fd9b55fe5370fc1bb746f"
server_key_version: "encryptionkey@0"
I20260809 06:25:42.276873 16233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:42.283425 16233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:42.284341 16233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:42.284679 16233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:42.303998 16233 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:37549
I20260809 06:25:42.304034 16336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:37549 every 8 connection(s)
I20260809 06:25:42.305456 16233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:42.309285 16337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:42.313786 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 16233
I20260809 06:25:42.314006 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:42.314677 8716 external_mini_cluster.cc:1428] Setting key 9b241412e1eda6a3ae1ba2e05ca4497c
I20260809 06:25:42.321538 16337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466: Bootstrap starting.
I20260809 06:25:42.324570 16342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:42.316840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.8.131.62:37549)
0809 06:25:42.318025 (+ 1185us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:42.318055 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:42.318114 (+ 59us) server_negotiation.cc:407] Connection header received
0809 06:25:42.320522 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:42.320549 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:42.321057 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:42.321499 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:42.322526 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:42.322534 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:42.322550 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:42.322574 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:42.322643 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:42.322665 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:42.322877 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:42.323032 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 06:25:42.323332 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":378,"threads_started":1}
I20260809 06:25:42.325449 16337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:42.326805 16337 log.cc:824] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:42.329636 16337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466: No bootstrap required, opened a new log
I20260809 06:25:42.337150 16337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d687b7d9b643d2a13845dbb458d466" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37549 } }
I20260809 06:25:42.337486 16337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:42.337558 16337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d687b7d9b643d2a13845dbb458d466, State: Initialized, Role: FOLLOWER
I20260809 06:25:42.338104 16337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d687b7d9b643d2a13845dbb458d466" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37549 } }
I20260809 06:25:42.338363 16337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:42.338478 16337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:42.338605 16337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:42.341168 16337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d687b7d9b643d2a13845dbb458d466" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37549 } }
I20260809 06:25:42.341609 16337 leader_election.cc:302] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98d687b7d9b643d2a13845dbb458d466; no voters:
I20260809 06:25:42.342175 16337 leader_election.cc:288] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:42.342338 16345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:42.343096 16345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [term 1 LEADER]: Becoming Leader. State: Replica: 98d687b7d9b643d2a13845dbb458d466, State: Running, Role: LEADER
I20260809 06:25:42.343693 16345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d687b7d9b643d2a13845dbb458d466" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37549 } }
I20260809 06:25:42.344544 16337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:42.346741 16350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98d687b7d9b643d2a13845dbb458d466. Latest consensus state: current_term: 1 leader_uuid: "98d687b7d9b643d2a13845dbb458d466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d687b7d9b643d2a13845dbb458d466" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37549 } } }
I20260809 06:25:42.347242 16350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:42.347724 16347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98d687b7d9b643d2a13845dbb458d466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d687b7d9b643d2a13845dbb458d466" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 37549 } } }
I20260809 06:25:42.348069 16347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:42.351249 16362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:42.356271 16362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:42.365181 16362 catalog_manager.cc:1347] Generated new cluster ID: 2b06db920ee041649927e93e1b90a265
I20260809 06:25:42.365309 16362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:42.381304 16362 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:42.381719 16362 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:42.389113 16362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466: Generated new TSK 0
I20260809 06:25:42.389822 16362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:25:42.396199 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:37549
--builtin_ntp_servers=127.8.131.20:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 06:25:42.752970 16377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:42.753242 16377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:42.753412 16377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:42.760859 16377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:42.761008 16377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:42.761220 16377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:42.771438 16377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:37549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:42.772907 16377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:42.774950 16377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:42.789064 16385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:42.789239 16387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:42.789892 16392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:42.790417 16377 server_base.cc:1034] running on GCE node
I20260809 06:25:42.791079 16377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:42.792121 16377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:42.793352 16377 hybrid_clock.cc:648] HybridClock initialized: now 1786256742793313 us; error 69 us; skew 500 ppm
I20260809 06:25:42.793687 16377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:25:42.796005 16377 webserver.cc:460] Webserver started at http://127.8.131.1:41553/ using document root <none> and password file <none>
I20260809 06:25:42.796707 16377 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:42.796854 16377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:42.797187 16377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:42.800166 16377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "e3651d102b1a4db5b2cd1bff83c6031d"
format_stamp: "Formatted at 2026-08-09 06:25:42 on dist-test-slave-mjjr"
server_key: "4decf79523ffc016817cae01a66ccc21"
server_key_iv: "4e68fd1ed9929c8874debb306e539b83"
server_key_version: "encryptionkey@0"
I20260809 06:25:42.801076 16377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "e3651d102b1a4db5b2cd1bff83c6031d"
format_stamp: "Formatted at 2026-08-09 06:25:42 on dist-test-slave-mjjr"
server_key: "4decf79523ffc016817cae01a66ccc21"
server_key_iv: "4e68fd1ed9929c8874debb306e539b83"
server_key_version: "encryptionkey@0"
I20260809 06:25:42.806246 16377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:42.809644 16400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:42.810956 16377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 06:25:42.811127 16377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "e3651d102b1a4db5b2cd1bff83c6031d"
format_stamp: "Formatted at 2026-08-09 06:25:42 on dist-test-slave-mjjr"
server_key: "4decf79523ffc016817cae01a66ccc21"
server_key_iv: "4e68fd1ed9929c8874debb306e539b83"
server_key_version: "encryptionkey@0"
I20260809 06:25:42.811363 16377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:42.817044 16377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:42.817838 16377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:42.818123 16377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:42.819018 16377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:42.820451 16377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:42.820572 16377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:42.820703 16377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:42.820817 16377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:42.855851 16377 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:40895
I20260809 06:25:42.855906 16562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:40895 every 8 connection(s)
I20260809 06:25:42.857268 16377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:42.861958 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 16377
I20260809 06:25:42.862190 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:42.862859 8716 external_mini_cluster.cc:1428] Setting key 67c6ddbf09d5ea3cab56842b8c46e60b
I20260809 06:25:42.865856 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:42.859854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:44317 (local address 127.8.131.62:37549)
0809 06:25:42.860308 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:42.860312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:42.861369 (+ 1057us) server_negotiation.cc:407] Connection header received
0809 06:25:42.863198 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:42.863203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:42.863279 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:42.863403 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:42.864979 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:42.864986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:42.864990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:42.865009 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:42.865061 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:42.865077 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:42.865204 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:42.865447 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 06:25:42.865511 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":250,"threads_started":1}
I20260809 06:25:42.866011 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:37549
--builtin_ntp_servers=127.8.131.20:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 06:25:42.866843 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:42.860304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37549 (local address 127.8.131.1:44317)
0809 06:25:42.861194 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:42.861251 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:42.862893 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:42.863661 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:42.863674 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:42.863791 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:42.864605 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:25:42.864609 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:42.864663 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:25:42.864780 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:42.865237 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:42.865255 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:42.865261 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 06:25:42.865481 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 06:25:42.865681 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":280,"threads_started":1}
I20260809 06:25:42.868388 16563 heartbeater.cc:347] Connected to a master server at 127.8.131.62:37549
I20260809 06:25:42.868686 16563 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:42.869292 16563 heartbeater.cc:511] Master 127.8.131.62:37549 requested a full tablet report, sending...
I20260809 06:25:42.871366 16285 ts_manager.cc:194] Registered new tserver with Master: e3651d102b1a4db5b2cd1bff83c6031d (127.8.131.1:40895)
W20260809 06:25:43.198112 16569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:43.198392 16569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:43.198484 16569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:43.205602 16569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:43.205699 16569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:43.205806 16569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:43.214784 16569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:37549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:43.216095 16569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:43.217893 16569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:43.231341 16582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:43.231297 16585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:43.231383 16579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:43.237202 16569 server_base.cc:1034] running on GCE node
I20260809 06:25:43.237929 16569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:43.238946 16569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:43.240208 16569 hybrid_clock.cc:648] HybridClock initialized: now 1786256743240148 us; error 45 us; skew 500 ppm
I20260809 06:25:43.240547 16569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:25:43.242841 16569 webserver.cc:460] Webserver started at http://127.8.131.2:45947/ using document root <none> and password file <none>
I20260809 06:25:43.243549 16569 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:43.243702 16569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:43.244074 16569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:43.247160 16569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83"
format_stamp: "Formatted at 2026-08-09 06:25:43 on dist-test-slave-mjjr"
server_key: "00975090fefb2452afb05bfb504be42a"
server_key_iv: "b3e843750ded7429903a57da5b321f32"
server_key_version: "encryptionkey@0"
I20260809 06:25:43.248011 16569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83"
format_stamp: "Formatted at 2026-08-09 06:25:43 on dist-test-slave-mjjr"
server_key: "00975090fefb2452afb05bfb504be42a"
server_key_iv: "b3e843750ded7429903a57da5b321f32"
server_key_version: "encryptionkey@0"
I20260809 06:25:43.253646 16569 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 06:25:43.257239 16593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:43.258586 16569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:25:43.258766 16569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83"
format_stamp: "Formatted at 2026-08-09 06:25:43 on dist-test-slave-mjjr"
server_key: "00975090fefb2452afb05bfb504be42a"
server_key_iv: "b3e843750ded7429903a57da5b321f32"
server_key_version: "encryptionkey@0"
I20260809 06:25:43.259004 16569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:43.264791 16569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:43.265643 16569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:43.265921 16569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:43.266858 16569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:43.268352 16569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:43.268491 16569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:43.268615 16569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:43.268703 16569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:43.303114 16569 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:37641
I20260809 06:25:43.303165 16759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:37641 every 8 connection(s)
I20260809 06:25:43.304848 16569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:43.309118 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 16569
I20260809 06:25:43.309288 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:43.309870 8716 external_mini_cluster.cc:1428] Setting key 2abd7abad4d10e78859a71d17a61ce00
I20260809 06:25:43.313138 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:37549
--builtin_ntp_servers=127.8.131.20:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 06:25:43.314816 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.307728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:35603 (local address 127.8.131.62:37549)
0809 06:25:43.307920 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.307925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.309669 (+ 1744us) server_negotiation.cc:407] Connection header received
0809 06:25:43.311875 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.311880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.311971 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.312052 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.313963 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.313969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.313975 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.313994 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.314035 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.314050 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.314164 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.314422 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.314482 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:25:43.316258 16763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.308293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37549 (local address 127.8.131.2:35603)
0809 06:25:43.309508 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:43.309553 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:43.311542 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:43.312549 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:43.312560 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:43.312672 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:43.313591 (+ 919us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:25:43.313597 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:43.313646 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:25:43.313773 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:43.314164 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:43.314177 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:43.314189 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 06:25:43.314494 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 06:25:43.314839 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":384,"threads_started":1}
I20260809 06:25:43.317677 16760 heartbeater.cc:347] Connected to a master server at 127.8.131.62:37549
I20260809 06:25:43.317988 16760 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:43.318692 16760 heartbeater.cc:511] Master 127.8.131.62:37549 requested a full tablet report, sending...
I20260809 06:25:43.320390 16285 ts_manager.cc:194] Registered new tserver with Master: 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 (127.8.131.2:37641)
W20260809 06:25:43.661974 16764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:43.662313 16764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:43.662417 16764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:43.669966 16764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:43.670068 16764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:43.670177 16764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:43.679469 16764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:37549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:43.680785 16764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:43.682746 16764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:43.696161 16770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:43.696134 16771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:43.696308 16773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:43.697458 16764 server_base.cc:1034] running on GCE node
I20260809 06:25:43.698086 16764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:43.699455 16764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:43.700680 16764 hybrid_clock.cc:648] HybridClock initialized: now 1786256743700624 us; error 53 us; skew 500 ppm
I20260809 06:25:43.701046 16764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:25:43.703388 16764 webserver.cc:460] Webserver started at http://127.8.131.3:35041/ using document root <none> and password file <none>
I20260809 06:25:43.704183 16764 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:43.704304 16764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:43.704650 16764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:43.707548 16764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "740191b5a0614bfaac35de606889798b"
format_stamp: "Formatted at 2026-08-09 06:25:43 on dist-test-slave-mjjr"
server_key: "45399cefa2ed06be235e2c8e90013aa7"
server_key_iv: "857dc0ffb3fc85167c5da3eddbc4c481"
server_key_version: "encryptionkey@0"
I20260809 06:25:43.708441 16764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "740191b5a0614bfaac35de606889798b"
format_stamp: "Formatted at 2026-08-09 06:25:43 on dist-test-slave-mjjr"
server_key: "45399cefa2ed06be235e2c8e90013aa7"
server_key_iv: "857dc0ffb3fc85167c5da3eddbc4c481"
server_key_version: "encryptionkey@0"
I20260809 06:25:43.713629 16764 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:43.717262 16781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:43.718586 16764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:25:43.718760 16764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "740191b5a0614bfaac35de606889798b"
format_stamp: "Formatted at 2026-08-09 06:25:43 on dist-test-slave-mjjr"
server_key: "45399cefa2ed06be235e2c8e90013aa7"
server_key_iv: "857dc0ffb3fc85167c5da3eddbc4c481"
server_key_version: "encryptionkey@0"
I20260809 06:25:43.719004 16764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:43.724857 16764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:43.725646 16764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:43.725926 16764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:43.726853 16764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:43.728323 16764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:43.728437 16764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:43.728575 16764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:43.728672 16764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:43.762828 16764 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:39241
I20260809 06:25:43.762930 16942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:39241 every 8 connection(s)
I20260809 06:25:43.764274 16764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:43.771281 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 16764
I20260809 06:25:43.771481 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:43.772125 8716 external_mini_cluster.cc:1428] Setting key 6f13b6c588c72c94097406a4ba2b108d
I20260809 06:25:43.772240 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.766782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:56475 (local address 127.8.131.62:37549)
0809 06:25:43.766959 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.766963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.768317 (+ 1354us) server_negotiation.cc:407] Connection header received
0809 06:25:43.769959 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.769964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.770019 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.770108 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.771488 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.771492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.771495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.771511 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.771572 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.771588 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.771744 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.771942 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.771995 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:25:43.773293 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.767199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:37549 (local address 127.8.131.3:56475)
0809 06:25:43.768156 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:43.768196 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:43.769720 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:43.770301 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:43.770308 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:43.770392 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:43.771073 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:25:43.771079 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:43.771134 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:25:43.771245 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:43.771713 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:43.771723 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:43.771731 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 06:25:43.772025 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 06:25:43.772283 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":287,"threads_started":1}
I20260809 06:25:43.774673 16944 heartbeater.cc:347] Connected to a master server at 127.8.131.62:37549
I20260809 06:25:43.774997 16944 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:43.775710 16944 heartbeater.cc:511] Master 127.8.131.62:37549 requested a full tablet report, sending...
I20260809 06:25:43.777103 16285 ts_manager.cc:194] Registered new tserver with Master: 740191b5a0614bfaac35de606889798b (127.8.131.3:39241)
I20260809 06:25:43.787415 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:25:43.794230 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.791938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.8.131.62:37549)
0809 06:25:43.792095 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.792099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.792610 (+ 511us) server_negotiation.cc:407] Connection header received
0809 06:25:43.792731 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.792734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.792787 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.792865 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.793681 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.793685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.793688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.793702 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.793733 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.793744 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.793836 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.793888 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.793933 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:25:43.798069 16285 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:25:43.805080 16285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:25:43.824996 16963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.817329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:39241 (local address 127.0.0.1:37128)
0809 06:25:43.819691 (+ 2362us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:43.819715 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:43.819830 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:43.821570 (+ 1740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:43.821582 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:43.821704 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:43.822753 (+ 1049us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:25:43.822759 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:43.822799 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:25:43.822922 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:43.823555 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:43.823569 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:43.823576 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 06:25:43.823825 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 06:25:43.823934 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2184,"thread_start_us":422,"threads_started":1}
I20260809 06:25:43.825481 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.818020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37128 (local address 127.8.131.3:39241)
0809 06:25:43.820816 (+ 2796us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.820830 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.820876 (+ 46us) server_negotiation.cc:407] Connection header received
0809 06:25:43.820982 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.820994 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.821299 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.821491 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.823298 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.823310 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.823323 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.823343 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.823405 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.823429 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.824857 (+ 1428us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.824943 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.825052 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2607,"thread_start_us":2434,"threads_started":1}
I20260809 06:25:43.826375 16958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.817178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:37641 (local address 127.0.0.1:53168)
0809 06:25:43.818877 (+ 1699us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:43.818933 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:43.819090 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:43.823801 (+ 4711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:43.823810 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:43.823834 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:43.824560 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:25:43.824564 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:43.824584 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:25:43.824636 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:43.825763 (+ 1127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:43.825768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:43.825772 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 06:25:43.825924 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 06:25:43.825983 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1442,"thread_start_us":1270,"threads_started":1}
I20260809 06:25:43.826992 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.819228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.8.131.1:40895)
0809 06:25:43.823524 (+ 4296us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.823533 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.823561 (+ 28us) server_negotiation.cc:407] Connection header received
0809 06:25:43.823634 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.823659 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.823854 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.824584 (+ 730us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.825829 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.825837 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.825876 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.825907 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.825961 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.825979 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.826523 (+ 544us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.826601 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.826694 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4126,"thread_start_us":3985,"threads_started":1}
I20260809 06:25:43.827345 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.817583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53168 (local address 127.8.131.2:37641)
0809 06:25:43.823258 (+ 5675us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.823267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.823296 (+ 29us) server_negotiation.cc:407] Connection header received
0809 06:25:43.823360 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.823366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.823592 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.824286 (+ 694us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.825520 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.825528 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.825541 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.825557 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.825618 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.825646 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.826893 (+ 1247us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.826957 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.827053 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5480,"thread_start_us":5332,"threads_started":1}
I20260809 06:25:43.827541 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.818704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:40895 (local address 127.0.0.1:50686)
0809 06:25:43.820129 (+ 1425us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:43.820146 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:43.820236 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:43.824069 (+ 3833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:43.824075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:43.824107 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:43.824782 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:25:43.824786 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:43.824802 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:25:43.824846 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:43.826114 (+ 1268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:43.826120 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:43.826125 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 06:25:43.826252 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 06:25:43.826308 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1321,"thread_start_us":724,"threads_started":1}
I20260809 06:25:43.829708 16847 tablet_service.cc:1467] Processing CreateTablet for tablet 95ddfad61452425eb01d2dba5dba0a63 (DEFAULT_TABLE table=test-table [id=5145a79d6d7c4d2d882b4a5bdd030b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:43.831001 16847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ddfad61452425eb01d2dba5dba0a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:43.831028 16661 tablet_service.cc:1467] Processing CreateTablet for tablet 95ddfad61452425eb01d2dba5dba0a63 (DEFAULT_TABLE table=test-table [id=5145a79d6d7c4d2d882b4a5bdd030b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:43.831693 16466 tablet_service.cc:1467] Processing CreateTablet for tablet 95ddfad61452425eb01d2dba5dba0a63 (DEFAULT_TABLE table=test-table [id=5145a79d6d7c4d2d882b4a5bdd030b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:25:43.832412 16661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ddfad61452425eb01d2dba5dba0a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:43.833042 16466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ddfad61452425eb01d2dba5dba0a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:43.842773 16965 tablet_bootstrap.cc:492] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: Bootstrap starting.
I20260809 06:25:43.846105 16965 tablet_bootstrap.cc:654] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:43.847734 16965 log.cc:824] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:43.849277 16966 tablet_bootstrap.cc:492] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: Bootstrap starting.
I20260809 06:25:43.848656 16967 tablet_bootstrap.cc:492] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: Bootstrap starting.
I20260809 06:25:43.852712 16967 tablet_bootstrap.cc:654] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:43.853086 16966 tablet_bootstrap.cc:654] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:43.853763 16967 log.cc:824] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:43.854302 16966 log.cc:824] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:43.854640 16965 tablet_bootstrap.cc:492] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: No bootstrap required, opened a new log
I20260809 06:25:43.854982 16965 ts_tablet_manager.cc:1397] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260809 06:25:43.856550 16967 tablet_bootstrap.cc:492] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: No bootstrap required, opened a new log
I20260809 06:25:43.856814 16967 ts_tablet_manager.cc:1397] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260809 06:25:43.860603 16966 tablet_bootstrap.cc:492] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: No bootstrap required, opened a new log
I20260809 06:25:43.860879 16966 ts_tablet_manager.cc:1397] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260809 06:25:43.865909 16965 raft_consensus.cc:348] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } }
I20260809 06:25:43.866580 16965 raft_consensus.cc:374] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:43.866757 16965 raft_consensus.cc:729] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 740191b5a0614bfaac35de606889798b, State: Initialized, Role: FOLLOWER
I20260809 06:25:43.867632 16965 consensus_queue.cc:260] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } }
I20260809 06:25:43.870736 16967 raft_consensus.cc:348] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } }
I20260809 06:25:43.871281 16967 raft_consensus.cc:374] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:43.871433 16967 raft_consensus.cc:729] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3651d102b1a4db5b2cd1bff83c6031d, State: Initialized, Role: FOLLOWER
I20260809 06:25:43.871429 16966 raft_consensus.cc:348] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } }
I20260809 06:25:43.871909 16966 raft_consensus.cc:374] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:43.872041 16966 raft_consensus.cc:729] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78f7c4a054dd4a2ba0a6bb4ef5b3bc83, State: Initialized, Role: FOLLOWER
I20260809 06:25:43.872254 16967 consensus_queue.cc:260] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } }
I20260809 06:25:43.872741 16966 consensus_queue.cc:260] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } }
I20260809 06:25:43.874367 16563 heartbeater.cc:503] Master 127.8.131.62:37549 was elected leader, sending a full tablet report...
I20260809 06:25:43.876287 16965 ts_tablet_manager.cc:1428] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
I20260809 06:25:43.876665 16944 heartbeater.cc:503] Master 127.8.131.62:37549 was elected leader, sending a full tablet report...
I20260809 06:25:43.884523 16977 raft_consensus.cc:482] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:25:43.884931 16977 raft_consensus.cc:504] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } }
I20260809 06:25:43.885197 16967 ts_tablet_manager.cc:1428] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: Time spent starting tablet: real 0.028s user 0.024s sys 0.000s
I20260809 06:25:43.885825 16760 heartbeater.cc:503] Master 127.8.131.62:37549 was elected leader, sending a full tablet report...
I20260809 06:25:43.886431 16966 ts_tablet_manager.cc:1428] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: Time spent starting tablet: real 0.025s user 0.012s sys 0.007s
I20260809 06:25:43.888121 16977 leader_election.cc:288] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 (127.8.131.2:37641), 740191b5a0614bfaac35de606889798b (127.8.131.3:39241)
I20260809 06:25:43.892815 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.887561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:47897 (local address 127.8.131.2:37641)
0809 06:25:43.887764 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.887770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.890182 (+ 2412us) server_negotiation.cc:407] Connection header received
0809 06:25:43.890477 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.890483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.890541 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.890611 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.892059 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.892065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.892068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.892087 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.892129 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.892147 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.892236 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.892476 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.892543 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:25:43.893047 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.887420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:37641 (local address 127.8.131.1:47897)
0809 06:25:43.890069 (+ 2649us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:43.890091 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:43.890301 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:43.890828 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:43.890832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:43.890860 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:43.891795 (+ 935us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:25:43.891801 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:43.891823 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:25:43.891891 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:43.892333 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:43.892339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:43.892346 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 06:25:43.892515 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 06:25:43.892566 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2521,"thread_start_us":996,"threads_started":1}
I20260809 06:25:43.893291 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.888657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:37919 (local address 127.8.131.3:39241)
0809 06:25:43.888854 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.888859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.891108 (+ 2249us) server_negotiation.cc:407] Connection header received
0809 06:25:43.891361 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.891367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.891445 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.891540 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.892657 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.892662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.892667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.892685 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.892727 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.892740 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.892881 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.892959 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.893025 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:25:43.893487 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.888988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:39241 (local address 127.8.131.1:37919)
0809 06:25:43.890988 (+ 2000us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:43.891013 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:43.891194 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:43.891725 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:43.891732 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:43.891755 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:43.892451 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:25:43.892453 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:43.892465 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:25:43.892509 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:43.892840 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:43.892844 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:43.892849 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 06:25:43.893014 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 06:25:43.893066 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1853,"thread_start_us":1274,"threads_started":1}
I20260809 06:25:43.894335 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ddfad61452425eb01d2dba5dba0a63" candidate_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" is_pre_election: true
I20260809 06:25:43.894362 16876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ddfad61452425eb01d2dba5dba0a63" candidate_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "740191b5a0614bfaac35de606889798b" is_pre_election: true
I20260809 06:25:43.894810 16688 raft_consensus.cc:2454] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3651d102b1a4db5b2cd1bff83c6031d in term 0.
I20260809 06:25:43.894837 16876 raft_consensus.cc:2454] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3651d102b1a4db5b2cd1bff83c6031d in term 0.
I20260809 06:25:43.895457 16403 leader_election.cc:302] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78f7c4a054dd4a2ba0a6bb4ef5b3bc83, e3651d102b1a4db5b2cd1bff83c6031d; no voters:
I20260809 06:25:43.895875 16977 raft_consensus.cc:2781] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:25:43.896028 16977 raft_consensus.cc:482] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:25:43.896164 16977 raft_consensus.cc:3037] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:43.898330 16977 raft_consensus.cc:504] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } }
I20260809 06:25:43.899024 16977 leader_election.cc:288] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [CANDIDATE]: Term 1 election: Requested vote from peers 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 (127.8.131.2:37641), 740191b5a0614bfaac35de606889798b (127.8.131.3:39241)
I20260809 06:25:43.899493 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ddfad61452425eb01d2dba5dba0a63" candidate_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83"
I20260809 06:25:43.899575 16876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ddfad61452425eb01d2dba5dba0a63" candidate_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "740191b5a0614bfaac35de606889798b"
I20260809 06:25:43.899765 16688 raft_consensus.cc:3037] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:43.899787 16876 raft_consensus.cc:3037] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:43.903195 16876 raft_consensus.cc:2454] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3651d102b1a4db5b2cd1bff83c6031d in term 1.
I20260809 06:25:43.903188 16688 raft_consensus.cc:2454] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3651d102b1a4db5b2cd1bff83c6031d in term 1.
I20260809 06:25:43.903627 16403 leader_election.cc:302] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 740191b5a0614bfaac35de606889798b, 78f7c4a054dd4a2ba0a6bb4ef5b3bc83, e3651d102b1a4db5b2cd1bff83c6031d; no voters:
I20260809 06:25:43.903971 16977 raft_consensus.cc:2781] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:43.904145 16977 raft_consensus.cc:686] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 1 LEADER]: Becoming Leader. State: Replica: e3651d102b1a4db5b2cd1bff83c6031d, State: Running, Role: LEADER
I20260809 06:25:43.904597 16977 consensus_queue.cc:237] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } }
I20260809 06:25:43.908478 16285 catalog_manager.cc:5495] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d reported cstate change: term changed from 0 to 1, leader changed from <none> to e3651d102b1a4db5b2cd1bff83c6031d (127.8.131.1). New cstate: current_term: 1 leader_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3651d102b1a4db5b2cd1bff83c6031d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 40895 } health_report { overall_health: HEALTHY } } }
I20260809 06:25:43.927465 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.925532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.8.131.1:40895)
0809 06:25:43.925695 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.925700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.925794 (+ 94us) server_negotiation.cc:407] Connection header received
0809 06:25:43.925974 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.925978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.926059 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.926149 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.926970 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.926974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.926978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.926992 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.927023 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.927034 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.927122 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.927195 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.927240 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:25:43.935842 16688 raft_consensus.cc:1261] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [term 1 FOLLOWER]: Refusing update from remote peer e3651d102b1a4db5b2cd1bff83c6031d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:43.935817 16876 raft_consensus.cc:1261] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [term 1 FOLLOWER]: Refusing update from remote peer e3651d102b1a4db5b2cd1bff83c6031d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:25:43.936833 16974 consensus_queue.cc:1035] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [LEADER]: Connected to new peer: Peer: permanent_uuid: "740191b5a0614bfaac35de606889798b" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:43.937206 16977 consensus_queue.cc:1035] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [LEADER]: Connected to new peer: Peer: permanent_uuid: "78f7c4a054dd4a2ba0a6bb4ef5b3bc83" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:43.949537 16987 mvcc.cc:205] Tried to move back new op lower bound from 7316507623150772224 to 7316507623036649472. Current Snapshot: MvccSnapshot[applied={T|T < 7316507623150772224}]
I20260809 06:25:43.968914 16285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:43.969208 16285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:25:43.972769 16285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98d687b7d9b643d2a13845dbb458d466: Sending DeleteTablet for 3 replicas of tablet 95ddfad61452425eb01d2dba5dba0a63
I20260809 06:25:43.974104 16661 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ddfad61452425eb01d2dba5dba0a63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:43 UTC) from {username='slave'} at 127.0.0.1:53168
I20260809 06:25:43.974113 16466 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ddfad61452425eb01d2dba5dba0a63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:43 UTC) from {username='slave'} at 127.0.0.1:50686
I20260809 06:25:43.974355 16847 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ddfad61452425eb01d2dba5dba0a63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:25:43 UTC) from {username='slave'} at 127.0.0.1:37128
I20260809 06:25:43.976328 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 16377
I20260809 06:25:43.986719 17007 tablet_replica.cc:330] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: stopping tablet replica
I20260809 06:25:43.987478 17007 raft_consensus.cc:2229] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:43.988277 17007 raft_consensus.cc:2258] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:43.989521 17006 tablet_replica.cc:330] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: stopping tablet replica
I20260809 06:25:43.990031 17006 raft_consensus.cc:2229] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:25:43.990799 17007 ts_tablet_manager.cc:1905] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:43.990738 17006 raft_consensus.cc:2258] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:43.992967 17006 ts_tablet_manager.cc:1905] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:43.998732 17007 ts_tablet_manager.cc:1918] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:25:43.998767 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:43.977810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.8.131.1:40895)
0809 06:25:43.977990 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:43.977996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:43.996128 (+ 18132us) server_negotiation.cc:407] Connection header received
0809 06:25:43.996361 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:43.996366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:43.996442 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:43.996551 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:43.997919 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:43.997925 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:43.997930 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:43.997954 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:43.997990 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:43.998006 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:43.998108 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:43.998359 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 06:25:43.998443 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:25:43.998910 17007 log.cc:1196] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/95ddfad61452425eb01d2dba5dba0a63
I20260809 06:25:43.999462 17007 ts_tablet_manager.cc:1939] T 95ddfad61452425eb01d2dba5dba0a63 P 78f7c4a054dd4a2ba0a6bb4ef5b3bc83: Deleting consensus metadata
I20260809 06:25:43.999914 16556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:44.001448 16260 catalog_manager.cc:4841] TS 78f7c4a054dd4a2ba0a6bb4ef5b3bc83 (127.8.131.2:37641): tablet 95ddfad61452425eb01d2dba5dba0a63 (table test-table [id=5145a79d6d7c4d2d882b4a5bdd030b86]) successfully deleted
I20260809 06:25:44.001591 17008 tablet_replica.cc:330] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: stopping tablet replica
I20260809 06:25:44.001993 17008 raft_consensus.cc:2229] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:44.002453 17008 raft_consensus.cc:2258] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:44.004709 17008 ts_tablet_manager.cc:1905] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:44.010401 17008 ts_tablet_manager.cc:1918] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:25:44.010608 17008 log.cc:1196] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/95ddfad61452425eb01d2dba5dba0a63
I20260809 06:25:44.011098 17008 ts_tablet_manager.cc:1939] T 95ddfad61452425eb01d2dba5dba0a63 P 740191b5a0614bfaac35de606889798b: Deleting consensus metadata
I20260809 06:25:44.012868 16258 catalog_manager.cc:4841] TS 740191b5a0614bfaac35de606889798b (127.8.131.3:39241): tablet 95ddfad61452425eb01d2dba5dba0a63 (table test-table [id=5145a79d6d7c4d2d882b4a5bdd030b86]) successfully deleted
I20260809 06:25:44.097003 17006 ts_tablet_manager.cc:1918] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:25:44.097369 17006 log.cc:1196] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/95ddfad61452425eb01d2dba5dba0a63
I20260809 06:25:44.097898 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 16377
I20260809 06:25:44.098063 17006 ts_tablet_manager.cc:1939] T 95ddfad61452425eb01d2dba5dba0a63 P e3651d102b1a4db5b2cd1bff83c6031d: Deleting consensus metadata
I20260809 06:25:44.124363 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 16569
W20260809 06:25:44.124317 16258 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.131.1:40895 (error 108)
I20260809 06:25:44.126296 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.125770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:40895 (local address 127.0.0.1:50722)
0809 06:25:44.125968 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:44.126122 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:40895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 06:25:44.126579 16258 catalog_manager.cc:4584] TS e3651d102b1a4db5b2cd1bff83c6031d (127.8.131.1:40895): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95ddfad61452425eb01d2dba5dba0a63: Network error: Client connection negotiation failed: client connection to 127.8.131.1:40895: connect: Connection refused (error 111)
I20260809 06:25:44.127574 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.125140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.8.131.2:37641)
0809 06:25:44.125322 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:44.125328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:44.125463 (+ 135us) server_negotiation.cc:407] Connection header received
0809 06:25:44.125690 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:44.125695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:44.125781 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:44.125915 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:44.126954 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:44.126958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:44.126961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:44.126976 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:44.127006 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:44.127018 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:44.127124 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:44.127211 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 06:25:44.127256 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 06:25:44.128207 16750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:44.180516 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.179894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:40895 (local address 127.0.0.1:50734)
0809 06:25:44.180138 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:44.180253 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:40895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 06:25:44.181735 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.181342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:40895 (local address 127.0.0.1:50740)
0809 06:25:44.181505 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:44.181588 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:40895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 06:25:44.215093 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 16569
I20260809 06:25:44.239777 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 16764
I20260809 06:25:44.242192 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.241588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:40895 (local address 127.0.0.1:50742)
0809 06:25:44.241852 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:44.241986 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:40895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 06:25:44.242914 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.240673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.8.131.3:39241)
0809 06:25:44.240863 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:44.240867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:44.240910 (+ 43us) server_negotiation.cc:407] Connection header received
0809 06:25:44.240967 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:44.240970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:44.241049 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:44.241156 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:44.242241 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:44.242245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:44.242248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:44.242264 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:44.242299 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:44.242328 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:44.242470 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:44.242569 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 06:25:44.242632 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:25:44.243311 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.242976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:40895 (local address 127.0.0.1:50758)
0809 06:25:44.243129 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:44.243204 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:40895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 06:25:44.243623 16935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:44.324107 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.323521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:40895 (local address 127.0.0.1:50768)
0809 06:25:44.323842 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:44.323957 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:40895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 06:25:44.325186 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.324780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:40895 (local address 127.0.0.1:50776)
0809 06:25:44.324996 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:44.325066 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:40895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 06:25:44.332577 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 16764
I20260809 06:25:44.357471 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 16233
I20260809 06:25:44.358628 16330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:44.466542 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 16233
2026-08-09T06:25:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2652 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 06:25:44.516201 8716 mini_oidc.cc:152] Starting JWKS server
I20260809 06:25:44.518949 8716 webserver.cc:460] Webserver started at https://127.0.0.1:40279/ using document root <none> and password file <none>
I20260809 06:25:44.519284 8716 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 06:25:44.520001 8716 webserver.cc:460] Webserver started at http://127.0.0.1:44819/ using document root <none> and password file <none>
2026-08-09T06:25:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:44Z Disabled control of system clock
I20260809 06:25:44.552225 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45007
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:32997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:45007
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40279/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/testchainca.cert with env {}
W20260809 06:25:44.884927 17024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:44.885259 17024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:44.885424 17024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:44.892670 17024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:44.892808 17024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:44.892879 17024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:44.892990 17024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:44.893078 17024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 06:25:44.893193 17024 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 06:25:44.893280 17024 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40279/jwks.json
I20260809 06:25:44.903362 17024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:32997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:45007
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40279/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:44.904814 17024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:44.906145 17024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:44.918310 17035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:44.918331 17041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:44.918248 17034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:44.919508 17024 server_base.cc:1034] running on GCE node
I20260809 06:25:44.920406 17024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:44.921856 17024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:44.923209 17024 hybrid_clock.cc:648] HybridClock initialized: now 1786256744923170 us; error 47 us; skew 500 ppm
I20260809 06:25:44.923604 17024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:25:44.925976 17024 webserver.cc:460] Webserver started at http://127.8.131.62:42937/ using document root <none> and password file <none>
I20260809 06:25:44.926697 17024 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:44.926856 17024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:44.927181 17024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:44.930184 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "925126ad73404b41888922fa465cef6e"
format_stamp: "Formatted at 2026-08-09 06:25:44 on dist-test-slave-mjjr"
server_key: "b3f5e8298511231671f2f02efd5e68fd"
server_key_iv: "a97ae31fdbe5dccea59dae0d614c5c09"
server_key_version: "encryptionkey@0"
I20260809 06:25:44.931020 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "925126ad73404b41888922fa465cef6e"
format_stamp: "Formatted at 2026-08-09 06:25:44 on dist-test-slave-mjjr"
server_key: "b3f5e8298511231671f2f02efd5e68fd"
server_key_iv: "a97ae31fdbe5dccea59dae0d614c5c09"
server_key_version: "encryptionkey@0"
I20260809 06:25:44.936342 17024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:25:44.939942 17051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:44.941279 17024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:25:44.941493 17024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "925126ad73404b41888922fa465cef6e"
format_stamp: "Formatted at 2026-08-09 06:25:44 on dist-test-slave-mjjr"
server_key: "b3f5e8298511231671f2f02efd5e68fd"
server_key_iv: "a97ae31fdbe5dccea59dae0d614c5c09"
server_key_version: "encryptionkey@0"
I20260809 06:25:44.941727 17024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:44.969623 17024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:44.970539 17024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:44.970906 17024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:44.990253 17024 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:45007
I20260809 06:25:44.990272 17132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:45007 every 8 connection(s)
I20260809 06:25:44.991742 17024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:44.993891 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 17024
I20260809 06:25:44.994125 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:44.994784 8716 external_mini_cluster.cc:1428] Setting key 99dfc203af3b093c5bd8da04d77442d7
I20260809 06:25:44.995898 17135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:45.010161 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:44.996829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60454 (local address 127.8.131.62:45007)
0809 06:25:44.997802 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:44.997814 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:44.997862 (+ 48us) server_negotiation.cc:407] Connection header received
0809 06:25:44.999839 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:44.999865 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:45.000433 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:45.000820 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:45.001639 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.003097 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.005174 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.005880 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.006024 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:45.007494 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:45.007502 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:45.007521 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:45.008273 (+ 752us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:45.008362 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:45.008388 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:45.008494 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:45.008759 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 06:25:45.009169 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":297,"threads_started":1}
I20260809 06:25:45.010867 17135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e: Bootstrap starting.
I20260809 06:25:45.014407 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:25:45.014614 17135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:45.015635 17135 log.cc:824] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:45.018844 17135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e: No bootstrap required, opened a new log
I20260809 06:25:45.026430 17135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "925126ad73404b41888922fa465cef6e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45007 } }
I20260809 06:25:45.026880 17135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:45.027007 17135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 925126ad73404b41888922fa465cef6e, State: Initialized, Role: FOLLOWER
I20260809 06:25:45.027614 17135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "925126ad73404b41888922fa465cef6e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45007 } }
I20260809 06:25:45.027926 17135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:45.028064 17135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:45.028255 17135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:45.030561 17135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "925126ad73404b41888922fa465cef6e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45007 } }
I20260809 06:25:45.031265 17135 leader_election.cc:302] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 925126ad73404b41888922fa465cef6e; no voters:
I20260809 06:25:45.032008 17135 leader_election.cc:288] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:45.032176 17144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:45.033316 17144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [term 1 LEADER]: Becoming Leader. State: Replica: 925126ad73404b41888922fa465cef6e, State: Running, Role: LEADER
I20260809 06:25:45.033870 17144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "925126ad73404b41888922fa465cef6e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45007 } }
I20260809 06:25:45.034227 17135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:45.036865 17146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 925126ad73404b41888922fa465cef6e. Latest consensus state: current_term: 1 leader_uuid: "925126ad73404b41888922fa465cef6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "925126ad73404b41888922fa465cef6e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45007 } } }
I20260809 06:25:45.037392 17146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:45.036907 17145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "925126ad73404b41888922fa465cef6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "925126ad73404b41888922fa465cef6e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45007 } } }
I20260809 06:25:45.038225 17145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:45.042174 17159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:45.047077 17159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:45.055308 17159 catalog_manager.cc:1347] Generated new cluster ID: 572f5a5cd7d34f7887f8e55818badac8
I20260809 06:25:45.055454 17159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:45.069707 17159 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:45.071048 17159 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:45.078754 17159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 925126ad73404b41888922fa465cef6e: Generated new TSK 0
I20260809 06:25:45.079612 17159 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 06:25:45.120666 17181 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 06:25:45.120906 17181 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 06:25:45.121877 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:45.100105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.8.131.62:45007)
0809 06:25:45.100356 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:45.100362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:45.100802 (+ 440us) server_negotiation.cc:407] Connection header received
0809 06:25:45.101234 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:45.101238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:45.101308 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:45.101436 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0809 06:25:45.102109 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.102836 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.104999 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.105439 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.105533 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:45.106170 (+ 637us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 06:25:45.121344 (+ 15174us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 06:25:45.121632 (+ 288us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60456: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 06:25:45.122030 17139 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60456: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 06:25:45.124017 17180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:25:45.099989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45007 (local address 127.0.0.1:60456)
0809 06:25:45.100676 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:45.100702 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:45.100922 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:45.101499 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:45.101504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:45.101518 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0809 06:25:45.101927 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:45.101941 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.102997 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.103004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:45.104851 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:45.104866 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.105620 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.105626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:45.105810 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:45.105901 (+ 91us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 06:25:45.121643 (+ 15742us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 06:25:45.122022 (+ 379us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.131.62:45007: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":350,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.131.62:45007: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 06:25:45.125765 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 17024
I20260809 06:25:45.127123 17125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:45.226035 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 17024
2026-08-09T06:25:45Z chronyd exiting
I20260809 06:25:45.282594 8716 test_util.cc:175] -----------------------------------------------
I20260809 06:25:45.282742 8716 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786256699676692-8716-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (771 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 06:25:45.286579 8716 mini_oidc.cc:152] Starting JWKS server
I20260809 06:25:45.288458 8716 webserver.cc:460] Webserver started at https://127.0.0.1:33259/ using document root <none> and password file <none>
I20260809 06:25:45.288645 8716 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 06:25:45.289249 8716 webserver.cc:460] Webserver started at http://127.0.0.1:39585/ using document root <none> and password file <none>
2026-08-09T06:25:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:45Z Disabled control of system clock
I20260809 06:25:45.321874 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38557
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:38557
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33259/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/testchainca.cert with env {}
W20260809 06:25:45.654582 17190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:45.654891 17190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:45.655059 17190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:45.662328 17190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:45.662446 17190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:45.662545 17190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:45.662667 17190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:45.662734 17190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 06:25:45.662844 17190 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 06:25:45.662928 17190 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33259/jwks.json
I20260809 06:25:45.673059 17190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:38557
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33259/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:45.674474 17190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:45.675835 17190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:45.687872 17204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:45.688098 17199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:45.687927 17201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:45.689172 17190 server_base.cc:1034] running on GCE node
I20260809 06:25:45.690003 17190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:45.691463 17190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:45.692729 17190 hybrid_clock.cc:648] HybridClock initialized: now 1786256745692689 us; error 52 us; skew 500 ppm
I20260809 06:25:45.693110 17190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:25:45.698428 17190 webserver.cc:460] Webserver started at http://127.8.131.62:41615/ using document root <none> and password file <none>
I20260809 06:25:45.699148 17190 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:45.699292 17190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:45.699633 17190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:45.702709 17190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62"
format_stamp: "Formatted at 2026-08-09 06:25:45 on dist-test-slave-mjjr"
server_key: "963d2a38bad4661ab387fd4efbef2847"
server_key_iv: "1c6251f4be7e1fed1294cf470508427f"
server_key_version: "encryptionkey@0"
I20260809 06:25:45.703532 17190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62"
format_stamp: "Formatted at 2026-08-09 06:25:45 on dist-test-slave-mjjr"
server_key: "963d2a38bad4661ab387fd4efbef2847"
server_key_iv: "1c6251f4be7e1fed1294cf470508427f"
server_key_version: "encryptionkey@0"
I20260809 06:25:45.708819 17190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:25:45.712263 17215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:45.713653 17190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:45.713876 17190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62"
format_stamp: "Formatted at 2026-08-09 06:25:45 on dist-test-slave-mjjr"
server_key: "963d2a38bad4661ab387fd4efbef2847"
server_key_iv: "1c6251f4be7e1fed1294cf470508427f"
server_key_version: "encryptionkey@0"
I20260809 06:25:45.714103 17190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:45.738729 17190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:45.739620 17190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:45.739989 17190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:45.758376 17190 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:38557
I20260809 06:25:45.758395 17288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:38557 every 8 connection(s)
I20260809 06:25:45.759795 17190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:45.763450 17289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:45.763969 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 17190
I20260809 06:25:45.764153 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:45.764822 8716 external_mini_cluster.cc:1428] Setting key bc17001290fe4c3099add764d1c5026d
I20260809 06:25:45.776582 17289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62: Bootstrap starting.
I20260809 06:25:45.778656 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:45.766914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43526 (local address 127.8.131.62:38557)
0809 06:25:45.767982 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:45.767992 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:45.768038 (+ 46us) server_negotiation.cc:407] Connection header received
0809 06:25:45.770023 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:45.770045 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:45.770508 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:45.770929 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:45.771507 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.772832 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.774137 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.774852 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.775060 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:45.776368 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:45.776377 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:45.776391 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:45.776972 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:45.777052 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:45.777076 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:45.777205 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:45.777426 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 06:25:45.777734 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":339,"threads_started":1}
I20260809 06:25:45.780192 17289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:45.781389 17289 log.cc:824] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:45.782850 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:25:45.784834 17289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62: No bootstrap required, opened a new log
I20260809 06:25:45.792606 17289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38557 } }
I20260809 06:25:45.792934 17289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:45.793008 17289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d5181d7b7fe4d9697f513dfbc5d4f62, State: Initialized, Role: FOLLOWER
I20260809 06:25:45.793557 17289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38557 } }
I20260809 06:25:45.793761 17289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:45.793865 17289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:45.793985 17289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:45.796108 17289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38557 } }
I20260809 06:25:45.796721 17289 leader_election.cc:302] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d5181d7b7fe4d9697f513dfbc5d4f62; no voters:
I20260809 06:25:45.804064 17289 leader_election.cc:288] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:45.804188 17296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:45.805413 17296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [term 1 LEADER]: Becoming Leader. State: Replica: 3d5181d7b7fe4d9697f513dfbc5d4f62, State: Running, Role: LEADER
I20260809 06:25:45.806067 17296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38557 } }
I20260809 06:25:45.808029 17297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38557 } } }
I20260809 06:25:45.808550 17297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:45.808877 17289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:45.808895 17298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d5181d7b7fe4d9697f513dfbc5d4f62. Latest consensus state: current_term: 1 leader_uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5181d7b7fe4d9697f513dfbc5d4f62" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38557 } } }
I20260809 06:25:45.809316 17298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:45.810010 17303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:45.820683 17303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:45.829231 17303 catalog_manager.cc:1347] Generated new cluster ID: e27f25df37164293ae0ced3a027dcf7d
I20260809 06:25:45.829360 17303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:45.852068 17303 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:45.853909 17303 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:45.864791 17303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d5181d7b7fe4d9697f513dfbc5d4f62: Generated new TSK 0
I20260809 06:25:45.866014 17303 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 06:25:45.898768 17338 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 06:25:45.898912 17338 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 06:25:45.900122 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:45.880971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.8.131.62:38557)
0809 06:25:45.881169 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:45.881180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:45.881599 (+ 419us) server_negotiation.cc:407] Connection header received
0809 06:25:45.881880 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:45.881885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:45.881955 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:45.882104 (+ 149us) server_negotiation.cc:226] Negotiated authn=JWT
0809 06:25:45.882573 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.883276 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.884720 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.885182 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.885326 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:45.885790 (+ 464us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 06:25:45.899411 (+ 13621us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 06:25:45.899800 (+ 389us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43540: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 06:25:45.900259 17292 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43540: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 06:25:45.900319 17333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:25:45.880823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38557 (local address 127.0.0.1:43540)
0809 06:25:45.881497 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:45.881512 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:45.881671 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:45.882121 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:45.882126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:45.882138 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0809 06:25:45.882419 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:45.882427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.883427 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.883435 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:45.884573 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:45.884585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:45.885354 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:45.885360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:45.885516 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:45.885547 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 06:25:45.899635 (+ 14088us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 06:25:45.899926 (+ 291us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.131.62:38557: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":398,"threads_started":1}
I20260809 06:25:45.901821 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 17190
I20260809 06:25:45.902998 17281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:45.997218 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 17190
2026-08-09T06:25:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (771 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 06:25:46.058454 8716 mini_oidc.cc:152] Starting JWKS server
I20260809 06:25:46.060499 8716 webserver.cc:460] Webserver started at https://127.0.0.1:46729/ using document root <none> and password file <none>
I20260809 06:25:46.060768 8716 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 06:25:46.061391 8716 webserver.cc:460] Webserver started at http://127.0.0.1:42615/ using document root <none> and password file <none>
2026-08-09T06:25:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:46Z Disabled control of system clock
I20260809 06:25:46.094050 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:34785
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:34785
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46729/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 06:25:46.417858 17351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:46.418202 17351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:46.418360 17351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:46.425429 17351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:46.425567 17351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:46.425657 17351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:46.425768 17351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:46.425858 17351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 06:25:46.425951 17351 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 06:25:46.426030 17351 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46729/jwks.json
I20260809 06:25:46.434999 17351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:34785
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:34785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46729/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:46.436304 17351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:46.438064 17351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:46.450402 17363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:46.450541 17359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:46.450520 17361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:46.451215 17351 server_base.cc:1034] running on GCE node
I20260809 06:25:46.452069 17351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:46.453474 17351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:46.454707 17351 hybrid_clock.cc:648] HybridClock initialized: now 1786256746454659 us; error 67 us; skew 500 ppm
I20260809 06:25:46.455046 17351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:25:46.457311 17351 webserver.cc:460] Webserver started at http://127.8.131.62:39247/ using document root <none> and password file <none>
I20260809 06:25:46.457991 17351 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:46.458128 17351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:46.458468 17351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:46.461506 17351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "c22c949165d84dccb08a2ca950f7d318"
format_stamp: "Formatted at 2026-08-09 06:25:46 on dist-test-slave-mjjr"
server_key: "95b21345b66ffa1b4a35610242446867"
server_key_iv: "c9dc1122692050b01b6d8c004e69c1b0"
server_key_version: "encryptionkey@0"
I20260809 06:25:46.462347 17351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "c22c949165d84dccb08a2ca950f7d318"
format_stamp: "Formatted at 2026-08-09 06:25:46 on dist-test-slave-mjjr"
server_key: "95b21345b66ffa1b4a35610242446867"
server_key_iv: "c9dc1122692050b01b6d8c004e69c1b0"
server_key_version: "encryptionkey@0"
I20260809 06:25:46.467393 17351 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 06:25:46.470717 17371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:46.472134 17351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:46.472328 17351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "c22c949165d84dccb08a2ca950f7d318"
format_stamp: "Formatted at 2026-08-09 06:25:46 on dist-test-slave-mjjr"
server_key: "95b21345b66ffa1b4a35610242446867"
server_key_iv: "c9dc1122692050b01b6d8c004e69c1b0"
server_key_version: "encryptionkey@0"
I20260809 06:25:46.472556 17351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:46.512694 17351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:46.513592 17351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:46.513916 17351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:46.534889 17351 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:34785
I20260809 06:25:46.534929 17446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:34785 every 8 connection(s)
I20260809 06:25:46.536309 17351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:46.540078 17447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:46.543727 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 17351
I20260809 06:25:46.543968 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:46.544618 8716 external_mini_cluster.cc:1428] Setting key bf98396f9c45d031601f4b28686e424d
I20260809 06:25:46.552527 17447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318: Bootstrap starting.
I20260809 06:25:46.556066 17447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:46.557322 17447 log.cc:824] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:46.560608 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:46.546529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50588 (local address 127.8.131.62:34785)
0809 06:25:46.547620 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:46.547634 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:46.547713 (+ 79us) server_negotiation.cc:407] Connection header received
0809 06:25:46.550599 (+ 2886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:46.550626 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:46.551197 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:46.551604 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:46.552183 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:46.554236 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:46.555523 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:46.556655 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:46.556836 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:46.557989 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:46.558004 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:46.558025 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:46.558842 (+ 817us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:46.558929 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:46.558950 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:46.559035 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:46.559287 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 06:25:46.559612 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":351,"threads_started":1}
I20260809 06:25:46.560633 17447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318: No bootstrap required, opened a new log
I20260809 06:25:46.564248 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:25:46.568498 17447 raft_consensus.cc:348] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22c949165d84dccb08a2ca950f7d318" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 34785 } }
I20260809 06:25:46.568861 17447 raft_consensus.cc:374] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:46.568969 17447 raft_consensus.cc:729] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22c949165d84dccb08a2ca950f7d318, State: Initialized, Role: FOLLOWER
I20260809 06:25:46.569557 17447 consensus_queue.cc:260] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22c949165d84dccb08a2ca950f7d318" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 34785 } }
I20260809 06:25:46.569801 17447 raft_consensus.cc:388] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:46.569953 17447 raft_consensus.cc:482] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:46.570140 17447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:46.572391 17447 raft_consensus.cc:504] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22c949165d84dccb08a2ca950f7d318" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 34785 } }
I20260809 06:25:46.572952 17447 leader_election.cc:302] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c22c949165d84dccb08a2ca950f7d318; no voters:
I20260809 06:25:46.573679 17447 leader_election.cc:288] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:46.573782 17454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:46.574508 17454 raft_consensus.cc:686] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [term 1 LEADER]: Becoming Leader. State: Replica: c22c949165d84dccb08a2ca950f7d318, State: Running, Role: LEADER
I20260809 06:25:46.575065 17454 consensus_queue.cc:237] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22c949165d84dccb08a2ca950f7d318" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 34785 } }
I20260809 06:25:46.576004 17447 sys_catalog.cc:564] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:46.578100 17455 sys_catalog.cc:455] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c22c949165d84dccb08a2ca950f7d318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22c949165d84dccb08a2ca950f7d318" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 34785 } } }
I20260809 06:25:46.578707 17455 sys_catalog.cc:458] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:46.579243 17457 sys_catalog.cc:455] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c22c949165d84dccb08a2ca950f7d318. Latest consensus state: current_term: 1 leader_uuid: "c22c949165d84dccb08a2ca950f7d318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c22c949165d84dccb08a2ca950f7d318" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 34785 } } }
I20260809 06:25:46.579631 17457 sys_catalog.cc:458] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:46.580926 17469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:46.585413 17469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:46.594600 17469 catalog_manager.cc:1347] Generated new cluster ID: ce0b7613ef4b4387ba8c8a8a521a1f14
I20260809 06:25:46.594740 17469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:46.608389 17469 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:46.609555 17469 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:46.615978 17469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c22c949165d84dccb08a2ca950f7d318: Generated new TSK 0
I20260809 06:25:46.616739 17469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:25:46.639611 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:46.631200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.8.131.62:34785)
0809 06:25:46.631357 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:46.631361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:46.631938 (+ 577us) server_negotiation.cc:407] Connection header received
0809 06:25:46.632134 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:46.632140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:46.632230 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:46.632408 (+ 178us) server_negotiation.cc:226] Negotiated authn=JWT
0809 06:25:46.632815 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:46.633943 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:46.635240 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:46.635989 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:46.636141 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:46.636611 (+ 470us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 06:25:46.638979 (+ 2368us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 06:25:46.639293 (+ 314us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50590: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 06:25:46.639782 17452 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50590: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 06:25:46.641261 17496 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:25:46.631224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:34785 (local address 127.0.0.1:50590)
0809 06:25:46.631826 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:46.631841 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:46.631965 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:46.632421 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:46.632424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:46.632433 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 06:25:46.632668 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:46.632675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:46.634087 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:46.634091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:46.635096 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:46.635106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:46.636172 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:46.636176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:46.636308 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:46.636338 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 06:25:46.639184 (+ 2846us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 06:25:46.640864 (+ 1680us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.131.62:34785: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46729/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":1395,"thread_start_us":285,"threads_started":1}
I20260809 06:25:46.642231 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 17351
I20260809 06:25:46.643672 17441 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 06:25:46.651666 17497 webserver.cc:533] Webserver: sslize failed:
E20260809 06:25:46.651799 17497 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 06:25:46.735726 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 17351
2026-08-09T06:25:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (739 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T06:25:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:46Z Disabled control of system clock
I20260809 06:25:46.830595 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:35655
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:45645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:35655
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/testchainca.cert with env {}
W20260809 06:25:47.160960 17505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:47.161271 17505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:47.161360 17505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:47.168560 17505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:47.168684 17505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:47.168721 17505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:47.168772 17505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:47.168818 17505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 06:25:47.168889 17505 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 06:25:47.168929 17505 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 06:25:47.179003 17505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:45645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:35655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:35655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:47.180442 17505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:47.181695 17505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:47.194033 17512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:47.194011 17515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:47.193964 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:47.197161 17505 server_base.cc:1034] running on GCE node
I20260809 06:25:47.197924 17505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:47.199265 17505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:47.200498 17505 hybrid_clock.cc:648] HybridClock initialized: now 1786256747200459 us; error 57 us; skew 500 ppm
I20260809 06:25:47.200855 17505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:25:47.203162 17505 webserver.cc:460] Webserver started at http://127.8.131.62:36257/ using document root <none> and password file <none>
I20260809 06:25:47.203979 17505 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:47.204131 17505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:47.204478 17505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:47.207423 17505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3"
format_stamp: "Formatted at 2026-08-09 06:25:47 on dist-test-slave-mjjr"
server_key: "94ba592a218019aaa2d91aff7c7e2f4b"
server_key_iv: "2234e90a08ed37e9ffbbf82a51d4663a"
server_key_version: "encryptionkey@0"
I20260809 06:25:47.208328 17505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3"
format_stamp: "Formatted at 2026-08-09 06:25:47 on dist-test-slave-mjjr"
server_key: "94ba592a218019aaa2d91aff7c7e2f4b"
server_key_iv: "2234e90a08ed37e9ffbbf82a51d4663a"
server_key_version: "encryptionkey@0"
I20260809 06:25:47.214067 17505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:47.217514 17524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:47.218870 17505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:25:47.219067 17505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3"
format_stamp: "Formatted at 2026-08-09 06:25:47 on dist-test-slave-mjjr"
server_key: "94ba592a218019aaa2d91aff7c7e2f4b"
server_key_iv: "2234e90a08ed37e9ffbbf82a51d4663a"
server_key_version: "encryptionkey@0"
I20260809 06:25:47.219298 17505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:47.230201 17505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:47.231016 17505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:47.231308 17505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:47.250478 17505 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:35655
I20260809 06:25:47.250484 17603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:35655 every 8 connection(s)
I20260809 06:25:47.251983 17505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:47.255853 17604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:47.261240 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 17505
I20260809 06:25:47.261441 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:47.262144 8716 external_mini_cluster.cc:1428] Setting key be9073000baa338088f330d556540561
I20260809 06:25:47.267359 17604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3: Bootstrap starting.
I20260809 06:25:47.271378 17604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:47.272449 17604 log.cc:824] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:47.275305 17604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3: No bootstrap required, opened a new log
I20260809 06:25:47.277446 17608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:47.264204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.8.131.62:35655)
0809 06:25:47.265351 (+ 1147us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:47.265365 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:47.265416 (+ 51us) server_negotiation.cc:407] Connection header received
0809 06:25:47.267793 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:47.267820 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:47.268382 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:47.268801 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:47.269394 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:47.271372 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:47.272775 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:47.273676 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:47.273890 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:47.275060 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:47.275067 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:47.275080 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:25:47.275683 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:47.275788 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:25:47.275812 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:47.275948 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:47.276169 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 06:25:47.276483 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":420,"threads_started":1}
I20260809 06:25:47.281404 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:25:47.284168 17604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35655 } }
I20260809 06:25:47.284622 17604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:47.284734 17604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b45a4f5f5a43aaab2e3f0a6c9935c3, State: Initialized, Role: FOLLOWER
I20260809 06:25:47.285293 17604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35655 } }
I20260809 06:25:47.285506 17604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:47.285604 17604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:47.285733 17604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:47.287967 17604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35655 } }
I20260809 06:25:47.288537 17604 leader_election.cc:302] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68b45a4f5f5a43aaab2e3f0a6c9935c3; no voters:
I20260809 06:25:47.289225 17604 leader_election.cc:288] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:47.289317 17610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:47.290220 17610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [term 1 LEADER]: Becoming Leader. State: Replica: 68b45a4f5f5a43aaab2e3f0a6c9935c3, State: Running, Role: LEADER
I20260809 06:25:47.290771 17610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35655 } }
I20260809 06:25:47.291504 17604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:47.293783 17614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35655 } } }
I20260809 06:25:47.294273 17614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:47.296605 17616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68b45a4f5f5a43aaab2e3f0a6c9935c3. Latest consensus state: current_term: 1 leader_uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b45a4f5f5a43aaab2e3f0a6c9935c3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35655 } } }
I20260809 06:25:47.297001 17616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:47.297215 17625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:47.301999 17625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:47.310765 17625 catalog_manager.cc:1347] Generated new cluster ID: 962a4f1467244a439191e217e3b0b759
I20260809 06:25:47.310880 17625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:47.327665 17625 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:47.329464 17625 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:47.345247 17625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68b45a4f5f5a43aaab2e3f0a6c9935c3: Generated new TSK 0
I20260809 06:25:47.346163 17625 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:25:47.830689 17608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:47.377486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.8.131.62:35655)
0809 06:25:47.377722 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:47.377736 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:47.378160 (+ 424us) server_negotiation.cc:407] Connection header received
0809 06:25:47.378440 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:47.378444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:47.378524 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:47.378694 (+ 170us) server_negotiation.cc:226] Negotiated authn=JWT
0809 06:25:47.379232 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:47.380023 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:47.381422 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:47.381910 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:47.382100 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:47.382587 (+ 487us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 06:25:47.829794 (+447207us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 06:25:47.830307 (+ 513us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 06:25:47.830864 17608 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 06:25:47.830962 17648 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:25:47.377395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:35655 (local address 127.0.0.1:46366)
0809 06:25:47.378040 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:47.378057 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:47.378236 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:47.378706 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:47.378711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:47.378724 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 06:25:47.379064 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:47.379073 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:47.380184 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:47.380189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:47.381258 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:47.381271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:47.382126 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:47.382133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:47.382276 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:47.382309 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 06:25:47.830154 (+447845us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 06:25:47.830502 (+ 348us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.131.62:35655: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":373,"threads_started":1}
I20260809 06:25:47.832508 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 17505
I20260809 06:25:47.833801 17599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:47.943342 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 17505
2026-08-09T06:25:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1191 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (421 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (421 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:48 dist-test-slave-mjjr krb5kdc[17668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:48 dist-test-slave-mjjr krb5kdc[17668](info): set up 2 sockets
Aug 09 06:25:48 dist-test-slave-mjjr krb5kdc[17668](info): commencing operation
krb5kdc: starting...
W20260809 06:25:50.086912 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:50 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:25:50.327754 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38101
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:38101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:50.669788 17688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:50.670159 17688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:50.670349 17688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:50.677384 17688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:50.677541 17688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:50.677625 17688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:50.677745 17688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:50.677814 17688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:50.687434 17688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:38101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:50.688793 17688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:50.690093 17688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:50.702090 17693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:50.702008 17701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:50.702059 17697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:50.703198 17688 server_base.cc:1034] running on GCE node
I20260809 06:25:50.704137 17688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:50.705547 17688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:50.706784 17688 hybrid_clock.cc:648] HybridClock initialized: now 1786256750706735 us; error 62 us; skew 500 ppm
I20260809 06:25:50.707142 17688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:50 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256750, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:50.713354 17688 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:25:50.714973 17688 webserver.cc:460] Webserver started at http://127.8.131.62:45071/ using document root <none> and password file <none>
I20260809 06:25:50.715742 17688 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:50.715893 17688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:50.716221 17688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:50.719291 17688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "4a11eb92c8c44151b47bd5ffa36d3a09"
format_stamp: "Formatted at 2026-08-09 06:25:50 on dist-test-slave-mjjr"
server_key: "9c5298b94f582495e0c7b209c9ed1f95"
server_key_iv: "879bfa798facdfba531f9a7c6ef9b5d7"
server_key_version: "encryptionkey@0"
I20260809 06:25:50.720134 17688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "4a11eb92c8c44151b47bd5ffa36d3a09"
format_stamp: "Formatted at 2026-08-09 06:25:50 on dist-test-slave-mjjr"
server_key: "9c5298b94f582495e0c7b209c9ed1f95"
server_key_iv: "879bfa798facdfba531f9a7c6ef9b5d7"
server_key_version: "encryptionkey@0"
I20260809 06:25:50.725251 17688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:25:50.728848 17713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:50.730126 17688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:25:50.730315 17688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "4a11eb92c8c44151b47bd5ffa36d3a09"
format_stamp: "Formatted at 2026-08-09 06:25:50 on dist-test-slave-mjjr"
server_key: "9c5298b94f582495e0c7b209c9ed1f95"
server_key_iv: "879bfa798facdfba531f9a7c6ef9b5d7"
server_key_version: "encryptionkey@0"
I20260809 06:25:50.730523 17688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:50.752254 17688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:50.756222 17688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:50.756522 17688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:50.776093 17688 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:38101
I20260809 06:25:50.776126 17792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:38101 every 8 connection(s)
I20260809 06:25:50.777549 17688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:50.781447 17793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:50.781553 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 17688
I20260809 06:25:50.781757 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:50.782600 8716 external_mini_cluster.cc:1428] Setting key b678b29365720ebfcaed9823e3c735bf
I20260809 06:25:50.795872 17793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09: Bootstrap starting.
Aug 09 06:25:50 dist-test-slave-mjjr krb5kdc[17668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:50.799537 17793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:50.800654 17793 log.cc:824] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:50.803627 17793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09: No bootstrap required, opened a new log
I20260809 06:25:50.806630 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:50.784798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.8.131.62:38101)
0809 06:25:50.785663 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:50.785675 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:50.785722 (+ 47us) server_negotiation.cc:407] Connection header received
0809 06:25:50.787984 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:50.788006 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:50.788412 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:50.788744 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:50.790472 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:50.791854 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:50.793721 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:50.794397 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:50.794529 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:50.799245 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:50.799253 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:50.799267 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:50.799308 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:50.802494 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:50.803186 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:50.803193 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:50.803199 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:50.803369 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:50.803850 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:50.803857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:50.803861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:50.804406 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:50.804716 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:50.805211 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 06:25:50.805640 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":399,"threads_started":1}
I20260809 06:25:50.812068 17793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a11eb92c8c44151b47bd5ffa36d3a09" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38101 } }
I20260809 06:25:50.812438 17793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:50.812556 17793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a11eb92c8c44151b47bd5ffa36d3a09, State: Initialized, Role: FOLLOWER
I20260809 06:25:50.813144 17793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a11eb92c8c44151b47bd5ffa36d3a09" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38101 } }
I20260809 06:25:50.813403 17793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:50.813552 17793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:50.813731 17793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:50.816241 17793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a11eb92c8c44151b47bd5ffa36d3a09" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38101 } }
I20260809 06:25:50.816746 17793 leader_election.cc:302] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a11eb92c8c44151b47bd5ffa36d3a09; no voters:
I20260809 06:25:50.817370 17793 leader_election.cc:288] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:50.817517 17802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:50.818236 17802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [term 1 LEADER]: Becoming Leader. State: Replica: 4a11eb92c8c44151b47bd5ffa36d3a09, State: Running, Role: LEADER
I20260809 06:25:50.818791 17802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a11eb92c8c44151b47bd5ffa36d3a09" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38101 } }
I20260809 06:25:50.819475 17793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:50.821720 17805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a11eb92c8c44151b47bd5ffa36d3a09. Latest consensus state: current_term: 1 leader_uuid: "4a11eb92c8c44151b47bd5ffa36d3a09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a11eb92c8c44151b47bd5ffa36d3a09" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38101 } } }
I20260809 06:25:50.822178 17805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:50.823688 17803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a11eb92c8c44151b47bd5ffa36d3a09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a11eb92c8c44151b47bd5ffa36d3a09" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38101 } } }
I20260809 06:25:50.826336 17803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:50.827472 17819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:50.832546 17819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:50.840921 17819 catalog_manager.cc:1347] Generated new cluster ID: c9cf5425398846e0860be278beba4956
I20260809 06:25:50.841043 17819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:50.876644 17819 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:50.878675 17819 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:50.893198 17819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a11eb92c8c44151b47bd5ffa36d3a09: Generated new TSK 0
I20260809 06:25:50.894243 17819 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:25:50.955725 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38101
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:51.286968 17839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:51.287237 17839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:51.287341 17839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:51.294430 17839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:51.294543 17839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:51.294656 17839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:51.304284 17839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:51.305473 17839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:51.306804 17839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:51.320022 17845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:51.320144 17846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:51.320329 17848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:51.321182 17839 server_base.cc:1034] running on GCE node
I20260809 06:25:51.321754 17839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:51.322641 17839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:51.323846 17839 hybrid_clock.cc:648] HybridClock initialized: now 1786256751323782 us; error 41 us; skew 500 ppm
I20260809 06:25:51.324173 17839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:51 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256751, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:51.327698 17839 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:25:51.329252 17839 webserver.cc:460] Webserver started at http://127.8.131.1:38023/ using document root <none> and password file <none>
I20260809 06:25:51.329957 17839 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:51.330112 17839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:51.330446 17839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:51.333636 17839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "2834588e9cff4255bb3d958062b02409"
format_stamp: "Formatted at 2026-08-09 06:25:51 on dist-test-slave-mjjr"
server_key: "bc15fa69b2fd16dd46941600e1f40370"
server_key_iv: "aebe19561761e4d4bea9decba4c0e9a6"
server_key_version: "encryptionkey@0"
I20260809 06:25:51.334458 17839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "2834588e9cff4255bb3d958062b02409"
format_stamp: "Formatted at 2026-08-09 06:25:51 on dist-test-slave-mjjr"
server_key: "bc15fa69b2fd16dd46941600e1f40370"
server_key_iv: "aebe19561761e4d4bea9decba4c0e9a6"
server_key_version: "encryptionkey@0"
I20260809 06:25:51.339476 17839 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 06:25:51.342949 17858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:51.344434 17839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:51.344623 17839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "2834588e9cff4255bb3d958062b02409"
format_stamp: "Formatted at 2026-08-09 06:25:51 on dist-test-slave-mjjr"
server_key: "bc15fa69b2fd16dd46941600e1f40370"
server_key_iv: "aebe19561761e4d4bea9decba4c0e9a6"
server_key_version: "encryptionkey@0"
I20260809 06:25:51.344833 17839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:51.365653 17839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:51.369580 17839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:51.369899 17839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:51.370862 17839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:51.372177 17839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:51.372290 17839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:51.372447 17839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:51.372521 17839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:51.406069 17839 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:44021
I20260809 06:25:51.406159 18015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:44021 every 8 connection(s)
I20260809 06:25:51.407374 17839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:51.408852 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 17839
I20260809 06:25:51.409056 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:51.409866 8716 external_mini_cluster.cc:1428] Setting key 963fd04398d73cf76cbe3c2acbde295a
Aug 09 06:25:51 dist-test-slave-mjjr krb5kdc[17668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256751, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:51.430095 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:51.410352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:49537 (local address 127.8.131.62:38101)
0809 06:25:51.410822 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:51.410829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:51.411934 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 06:25:51.413708 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:51.413716 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:51.413803 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:51.413951 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:51.415365 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:51.416535 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:51.417993 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:51.418696 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:51.418858 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:51.422905 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:51.422914 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:51.422922 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:51.422989 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:51.426777 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:51.427479 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:51.427488 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:51.427495 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:51.427673 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:51.428099 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:51.428106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:51.428112 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:51.428474 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:51.428613 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:51.429322 (+ 709us) server_negotiation.cc:299] Negotiation successful
0809 06:25:51.429573 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":250,"threads_started":1}
I20260809 06:25:51.430619 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:51.410892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38101 (local address 127.8.131.1:49537)
0809 06:25:51.411751 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:51.411797 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:51.413434 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:51.414143 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:51.414151 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:51.414640 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:51.415195 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:51.415212 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:51.416687 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:51.416690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:51.417830 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:51.417842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:51.418925 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:51.418930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:51.419136 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:51.419856 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:51.419882 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:51.422674 (+ 2792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:51.426925 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:51.426931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:51.426941 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:51.427349 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:51.427804 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:51.427807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:51.427809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:51.427988 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:51.428595 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:51.428600 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:51.428885 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 06:25:51.429151 (+ 266us) client_negotiation.cc:240] Negotiation successful
0809 06:25:51.429528 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":363,"threads_started":1}
I20260809 06:25:51.432339 18018 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38101
I20260809 06:25:51.432680 18018 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:51.433666 18018 heartbeater.cc:511] Master 127.8.131.62:38101 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
I20260809 06:25:51.436252 17741 ts_manager.cc:194] Registered new tserver with Master: 2834588e9cff4255bb3d958062b02409 (127.8.131.1:44021)
I20260809 06:25:51.437685 17741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:49537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:25:51.464956 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38101
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:51.796588 18026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:51.796886 18026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:51.796983 18026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:51.804405 18026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:51.804526 18026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:51.804638 18026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:51.814636 18026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:51.816030 18026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:51.817324 18026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:51.830685 18035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:51.830755 18038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:51.830736 18036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:51.831715 18026 server_base.cc:1034] running on GCE node
I20260809 06:25:51.832365 18026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:51.833379 18026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:51.834623 18026 hybrid_clock.cc:648] HybridClock initialized: now 1786256751834562 us; error 55 us; skew 500 ppm
I20260809 06:25:51.834932 18026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:51 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256751, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:51.838168 18026 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:25:51.839511 18026 webserver.cc:460] Webserver started at http://127.8.131.2:45401/ using document root <none> and password file <none>
I20260809 06:25:51.840184 18026 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:51.840291 18026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:51.840549 18026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:51.843240 18026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "e6d2fc8066ff4f46a3d1acfc53367232"
format_stamp: "Formatted at 2026-08-09 06:25:51 on dist-test-slave-mjjr"
server_key: "b458c321b73fbab0950fe959fa9aec1e"
server_key_iv: "73640138c23f70becf8a5f2feb71ef10"
server_key_version: "encryptionkey@0"
I20260809 06:25:51.844058 18026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "e6d2fc8066ff4f46a3d1acfc53367232"
format_stamp: "Formatted at 2026-08-09 06:25:51 on dist-test-slave-mjjr"
server_key: "b458c321b73fbab0950fe959fa9aec1e"
server_key_iv: "73640138c23f70becf8a5f2feb71ef10"
server_key_version: "encryptionkey@0"
I20260809 06:25:51.849304 18026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:25:51.852744 18051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:51.854045 18026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:25:51.854225 18026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "e6d2fc8066ff4f46a3d1acfc53367232"
format_stamp: "Formatted at 2026-08-09 06:25:51 on dist-test-slave-mjjr"
server_key: "b458c321b73fbab0950fe959fa9aec1e"
server_key_iv: "73640138c23f70becf8a5f2feb71ef10"
server_key_version: "encryptionkey@0"
I20260809 06:25:51.854442 18026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:51.876062 18026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:51.880015 18026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:51.880259 18026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:51.881138 18026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:51.882411 18026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:51.882483 18026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:51.882570 18026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:51.882614 18026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:51.915586 18026 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:40261
I20260809 06:25:51.915632 18214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:40261 every 8 connection(s)
I20260809 06:25:51.916893 18026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:51.917366 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 18026
I20260809 06:25:51.917596 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:51.918185 8716 external_mini_cluster.cc:1428] Setting key 9e72e90b9d15909abf25c373d0b0c634
Aug 09 06:25:51 dist-test-slave-mjjr krb5kdc[17668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256751, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:51.940665 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:51.920068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:52867 (local address 127.8.131.62:38101)
0809 06:25:51.920253 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:51.920259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:51.921843 (+ 1584us) server_negotiation.cc:407] Connection header received
0809 06:25:51.924731 (+ 2888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:51.924735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:51.924805 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:51.924894 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:51.927005 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:51.927800 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:51.930291 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:51.930728 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:51.930867 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:51.934859 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:51.934866 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:51.934878 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:51.934918 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:51.937539 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:51.938257 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:51.938265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:51.938269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:51.938413 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:51.938848 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:51.938853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:51.938857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:51.939142 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:51.939302 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:51.940053 (+ 751us) server_negotiation.cc:299] Negotiation successful
0809 06:25:51.940301 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 06:25:51.941628 18222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:51.920716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38101 (local address 127.8.131.2:52867)
0809 06:25:51.921659 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:51.921715 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:51.924447 (+ 2732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:51.925164 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:51.925178 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:51.925989 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:51.926827 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:51.926855 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:51.927953 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:51.927961 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:51.930132 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:51.930148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:51.930978 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:51.930985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:51.931323 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:51.932111 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:51.932140 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:51.934636 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:51.937684 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:51.937692 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:51.937708 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:51.938132 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:51.938528 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:51.938533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:51.938537 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:51.938729 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:51.939274 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:51.939282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:51.939618 (+ 336us) client_negotiation.cc:769] Sending connection context
0809 06:25:51.939953 (+ 335us) client_negotiation.cc:240] Negotiation successful
0809 06:25:51.940497 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":361,"threads_started":1}
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
I20260809 06:25:51.943169 18218 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38101
I20260809 06:25:51.943452 18218 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:51.944300 18218 heartbeater.cc:511] Master 127.8.131.62:38101 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
I20260809 06:25:51.945902 17741 ts_manager.cc:194] Registered new tserver with Master: e6d2fc8066ff4f46a3d1acfc53367232 (127.8.131.2:40261)
I20260809 06:25:51.946756 17741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:52867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:25:51.970923 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38101
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:52.296995 18226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:52.297272 18226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:52.297461 18226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:52.304698 18226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:52.304828 18226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:52.305017 18226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:52.314713 18226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:52.316275 18226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:52.317570 18226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:52.331190 18234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:52.331151 18242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:52.331112 18235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:52.332238 18226 server_base.cc:1034] running on GCE node
I20260809 06:25:52.332950 18226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:52.334057 18226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:52.335285 18226 hybrid_clock.cc:648] HybridClock initialized: now 1786256752335236 us; error 44 us; skew 500 ppm
I20260809 06:25:52.335613 18226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:52.338943 18226 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:25:52.340461 18226 webserver.cc:460] Webserver started at http://127.8.131.3:33543/ using document root <none> and password file <none>
I20260809 06:25:52.341136 18226 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:52.341243 18226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:52.341598 18226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:52.344628 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "baec5fc39d1a417fafb111271e8cf6cc"
format_stamp: "Formatted at 2026-08-09 06:25:52 on dist-test-slave-mjjr"
server_key: "095ae57fd60b9352fdace0fcdcbf6714"
server_key_iv: "3da4bd2c6cf10f80dd2134f1cf26f9fe"
server_key_version: "encryptionkey@0"
I20260809 06:25:52.345358 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "baec5fc39d1a417fafb111271e8cf6cc"
format_stamp: "Formatted at 2026-08-09 06:25:52 on dist-test-slave-mjjr"
server_key: "095ae57fd60b9352fdace0fcdcbf6714"
server_key_iv: "3da4bd2c6cf10f80dd2134f1cf26f9fe"
server_key_version: "encryptionkey@0"
I20260809 06:25:52.350756 18226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:25:52.354132 18250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:52.355453 18226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:25:52.355588 18226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "baec5fc39d1a417fafb111271e8cf6cc"
format_stamp: "Formatted at 2026-08-09 06:25:52 on dist-test-slave-mjjr"
server_key: "095ae57fd60b9352fdace0fcdcbf6714"
server_key_iv: "3da4bd2c6cf10f80dd2134f1cf26f9fe"
server_key_version: "encryptionkey@0"
I20260809 06:25:52.355839 18226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:52.383882 18226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:52.387811 18226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:52.388063 18226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:52.388988 18226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:52.390293 18226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:52.390369 18226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:52.390465 18226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:52.390517 18226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:52.425899 18226 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:33197
I20260809 06:25:52.425935 18424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:33197 every 8 connection(s)
I20260809 06:25:52.427181 18226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:52.433430 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 18226
I20260809 06:25:52.433615 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:52.434134 8716 external_mini_cluster.cc:1428] Setting key 2370cf55fc21b978d786cad6f6954d3e
I20260809 06:25:52.440498 18018 heartbeater.cc:503] Master 127.8.131.62:38101 was elected leader, sending a full tablet report...
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:25:52.447496 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:52.429736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:58905 (local address 127.8.131.62:38101)
0809 06:25:52.429924 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.429928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.431059 (+ 1131us) server_negotiation.cc:407] Connection header received
0809 06:25:52.432838 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.432842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.432922 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.433010 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.434818 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.436007 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.437552 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.437984 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.438127 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:52.441648 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:52.441653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:52.441656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:52.441690 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:52.444329 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.445112 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.445117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.445122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.445260 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.445705 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.445713 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.445717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.446003 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:52.446174 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:52.446890 (+ 716us) server_negotiation.cc:299] Negotiation successful
0809 06:25:52.447118 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:25:52.448643 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:52.430182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38101 (local address 127.8.131.3:58905)
0809 06:25:52.430887 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:52.430929 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:52.432600 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:52.433252 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:52.433259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:52.433914 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:52.434609 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:52.434637 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.436189 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.436193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:52.437405 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:52.437417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.438225 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.438232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:52.438474 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:52.439086 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:52.439107 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:52.441423 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:52.444497 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:52.444507 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:52.444524 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:52.444974 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:52.445380 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:52.445385 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:52.445389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:52.445587 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:52.446145 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:52.446152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:52.446511 (+ 359us) client_negotiation.cc:769] Sending connection context
0809 06:25:52.446783 (+ 272us) client_negotiation.cc:240] Negotiation successful
0809 06:25:52.447347 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":296,"threads_started":1}
I20260809 06:25:52.450012 18426 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38101
I20260809 06:25:52.450330 18426 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:52.451048 18426 heartbeater.cc:511] Master 127.8.131.62:38101 requested a full tablet report, sending...
I20260809 06:25:52.452427 17741 ts_manager.cc:194] Registered new tserver with Master: baec5fc39d1a417fafb111271e8cf6cc (127.8.131.3:33197)
I20260809 06:25:52.453277 17741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:58905
I20260809 06:25:52.461897 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/test-admin.keytab.
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:52.493052 8716 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 06:25:52.498395 18436 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:25:52.495868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.8.131.1:44021)
0809 06:25:52.496501 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.496508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.496549 (+ 41us) server_negotiation.cc:407] Connection header received
0809 06:25:52.496607 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.496613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.496816 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.496986 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.497870 (+ 884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51832: BlockingRecv error: recv got EOF from 127.0.0.1:51832 (error 108)
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":317,"threads_started":1}
W20260809 06:25:52.498409 8716 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5cc' of type FILE
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:52.500313 8716 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:52.501727 8716 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:52.516814 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:52.503315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.8.131.1:44021)
0809 06:25:52.503521 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.503527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.503755 (+ 228us) server_negotiation.cc:407] Connection header received
0809 06:25:52.503901 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.503904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.503961 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.504044 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.504987 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.505986 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.507178 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.507924 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.508023 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:52.511588 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:52.511599 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:52.511612 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:52.511682 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:52.514350 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.514939 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.514946 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.514948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.515066 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.515385 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.515388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.515399 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.515808 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:52.516026 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:52.516350 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 06:25:52.516553 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260809 06:25:52.522629 18436 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:25:52.520761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.8.131.1:44021)
0809 06:25:52.520972 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.520980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.521236 (+ 256us) server_negotiation.cc:407] Connection header received
0809 06:25:52.521414 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.521419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.521481 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.521580 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.522359 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51844: BlockingRecv error: recv got EOF from 127.0.0.1:51844 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 06:25:52.522871 8716 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5cc' of type FILE
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:52.524572 8716 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:52.525945 8716 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:52.539553 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:52.527132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.8.131.1:44021)
0809 06:25:52.527303 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.527310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.527675 (+ 365us) server_negotiation.cc:407] Connection header received
0809 06:25:52.527841 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.527850 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.527907 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.528005 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.528864 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.529546 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.530774 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.531188 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.531326 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:52.534937 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:52.534942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:52.534945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:52.534974 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:52.537397 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.537985 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.537989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.537991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.538098 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.538417 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.538420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.538422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.538664 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:52.538845 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:52.539087 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 06:25:52.539268 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 06:25:52.543805 18436 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:25:52.542137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51866 (local address 127.8.131.1:44021)
0809 06:25:52.542333 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.542340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.542509 (+ 169us) server_negotiation.cc:407] Connection header received
0809 06:25:52.542678 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.542689 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.542754 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.542851 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.543541 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51866: BlockingRecv error: recv got EOF from 127.0.0.1:51866 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 06:25:52.544013 8716 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786256699676692-8716-0/krb5kdc/krb5cc' of type FILE
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:52.545465 8716 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:52.546774 8716 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:25:52.561345 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:52.548070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51870 (local address 127.8.131.1:44021)
0809 06:25:52.548256 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.548262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.548485 (+ 223us) server_negotiation.cc:407] Connection header received
0809 06:25:52.548672 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.548677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.548736 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.548833 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.549944 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.550666 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.552006 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.552444 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.552581 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:52.556165 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:52.556170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:52.556173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:52.556205 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:52.558730 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.559322 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.559327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.559329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.559483 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.559870 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.559873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.559875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.560162 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:52.560469 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:52.560811 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 06:25:52.561037 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 06:25:52.563000 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 17839
I20260809 06:25:52.574870 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:52.563893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51880 (local address 127.8.131.1:44021)
0809 06:25:52.564105 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.564110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.564353 (+ 243us) server_negotiation.cc:407] Connection header received
0809 06:25:52.564543 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.564552 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.564606 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.564704 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.565626 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.566289 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.567494 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.567966 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.568108 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:52.570062 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:52.570067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:52.570070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:52.570098 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:52.572402 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.573000 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.573009 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.573013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.573153 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.573521 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.573526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.573529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.573770 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:52.574047 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:52.574362 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 06:25:52.574564 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 06:25:52.575539 18009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:52.661104 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 17839
I20260809 06:25:52.687799 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 18026
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:25:52.704306 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:52.688927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.8.131.2:40261)
0809 06:25:52.689577 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.689585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.689613 (+ 28us) server_negotiation.cc:407] Connection header received
0809 06:25:52.689673 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.689679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.689895 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.690042 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.691069 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.692267 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.693546 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.694138 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.694247 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:52.698221 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:52.698232 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:52.698247 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:52.698286 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:52.701366 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.702078 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.702085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.702088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.702218 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.702617 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.702623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.702626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.703002 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:52.703320 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:52.703696 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 06:25:52.703984 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":349,"threads_started":1}
I20260809 06:25:52.705650 18210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:52.789004 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 18026
I20260809 06:25:52.811790 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 18226
Aug 09 06:25:52 dist-test-slave-mjjr krb5kdc[17668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:25:52.827606 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:52.812849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45870 (local address 127.8.131.3:33197)
0809 06:25:52.813524 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:52.813531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:52.813560 (+ 29us) server_negotiation.cc:407] Connection header received
0809 06:25:52.813620 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:52.813627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:52.813837 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:52.813981 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:52.815062 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.816322 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.817555 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:52.818108 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:52.818204 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:52.822188 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:52.822199 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:52.822213 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:52.822247 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:52.825119 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.825764 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.825770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.825773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.825895 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:52.826211 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:52.826214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:52.826216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:52.826565 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:52.826799 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:52.827024 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 06:25:52.827288 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":345,"threads_started":1}
I20260809 06:25:52.829012 18417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:52.913059 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 18226
I20260809 06:25:52.935436 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 17688
I20260809 06:25:52.936718 17785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:53.040802 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 17688
2026-08-09T06:25:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4256 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:25:53 dist-test-slave-mjjr krb5kdc[18470](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:25:53 dist-test-slave-mjjr krb5kdc[18470](info): set up 2 sockets
Aug 09 06:25:53 dist-test-slave-mjjr krb5kdc[18470](info): commencing operation
krb5kdc: starting...
W20260809 06:25:54.357930 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.009s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:25:54 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:25:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:25:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.62.keytab.
Entry for principal oryx/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.62.keytab.
I20260809 06:25:54.622731 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45687
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.62.keytab
--principal=oryx/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:45687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:54.965749 18484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:54.966066 18484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:54.966157 18484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:54.974005 18484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:54.974148 18484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:54.974210 18484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:54.974273 18484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:54.974319 18484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:54.984822 18484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:45687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.62.keytab
--principal=oryx/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:54.986238 18484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:54.987627 18484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:54.999955 18493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:54.999984 18492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:55.000780 18496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:55.001174 18484 server_base.cc:1034] running on GCE node
I20260809 06:25:55.002020 18484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:55.003480 18484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:55.004711 18484 hybrid_clock.cc:648] HybridClock initialized: now 1786256755004649 us; error 43 us; skew 500 ppm
I20260809 06:25:55.005112 18484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:55 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256755, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:55.008683 18484 init.cc:377] Logged in from keytab as oryx/127.8.131.62@KRBTEST.COM (short username oryx)
I20260809 06:25:55.010411 18484 webserver.cc:460] Webserver started at http://127.8.131.62:36759/ using document root <none> and password file <none>
I20260809 06:25:55.011185 18484 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:55.011372 18484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:55.011794 18484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:55.015203 18484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "467cc2d070a8428d9cd4281d718a7875"
format_stamp: "Formatted at 2026-08-09 06:25:55 on dist-test-slave-mjjr"
server_key: "76d667e51615d898aa6e1c38085c3052"
server_key_iv: "2f2f2bbe59643ff762ab5794818284db"
server_key_version: "encryptionkey@0"
I20260809 06:25:55.016163 18484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "467cc2d070a8428d9cd4281d718a7875"
format_stamp: "Formatted at 2026-08-09 06:25:55 on dist-test-slave-mjjr"
server_key: "76d667e51615d898aa6e1c38085c3052"
server_key_iv: "2f2f2bbe59643ff762ab5794818284db"
server_key_version: "encryptionkey@0"
I20260809 06:25:55.021895 18484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:25:55.025668 18507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:55.027073 18484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:25:55.027308 18484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "467cc2d070a8428d9cd4281d718a7875"
format_stamp: "Formatted at 2026-08-09 06:25:55 on dist-test-slave-mjjr"
server_key: "76d667e51615d898aa6e1c38085c3052"
server_key_iv: "2f2f2bbe59643ff762ab5794818284db"
server_key_version: "encryptionkey@0"
I20260809 06:25:55.027539 18484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:55.063900 18484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:55.068666 18484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:55.068986 18484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:55.087782 18484 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:45687
I20260809 06:25:55.087785 18578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:45687 every 8 connection(s)
I20260809 06:25:55.089344 18484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:25:55.093349 18580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:55.098800 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 18484
I20260809 06:25:55.099035 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:25:55.099905 8716 external_mini_cluster.cc:1428] Setting key 5cfc4dcf3c3ff2b28044361222761a78
I20260809 06:25:55.105505 18580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: Bootstrap starting.
I20260809 06:25:55.109329 18580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:55.110704 18580 log.cc:824] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:55.114372 18580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: No bootstrap required, opened a new log
Aug 09 06:25:55 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:25:55.126029 18580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } }
I20260809 06:25:55.126542 18580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:55.126662 18580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 467cc2d070a8428d9cd4281d718a7875, State: Initialized, Role: FOLLOWER
I20260809 06:25:55.127126 18584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:55.102271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49494 (local address 127.8.131.62:45687)
0809 06:25:55.103270 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:55.103286 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:55.103334 (+ 48us) server_negotiation.cc:407] Connection header received
0809 06:25:55.105303 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:55.105328 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:55.105817 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:55.106169 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:55.108141 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:55.109531 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:55.111381 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:55.112474 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:55.112608 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:55.117920 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:55.117930 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:55.117958 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:55.118011 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:55.122247 (+ 4236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:55.122939 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:55.122948 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:55.122958 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:55.123174 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:55.123587 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:55.123592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:55.123596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:55.124479 (+ 883us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:55.124793 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:55.125140 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 06:25:55.125606 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":500,"threads_started":1}
I20260809 06:25:55.127386 18580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } }
I20260809 06:25:55.127696 18580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:25:55.127861 18580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:25:55.128063 18580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:55.130852 18580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } }
I20260809 06:25:55.131301 18580 leader_election.cc:302] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 467cc2d070a8428d9cd4281d718a7875; no voters:
I20260809 06:25:55.132076 18580 leader_election.cc:288] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:25:55.132254 18586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:25:55.133035 18586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 1 LEADER]: Becoming Leader. State: Replica: 467cc2d070a8428d9cd4281d718a7875, State: Running, Role: LEADER
I20260809 06:25:55.133592 18586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } }
I20260809 06:25:55.134279 18580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:55.136909 18588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 467cc2d070a8428d9cd4281d718a7875. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } }
I20260809 06:25:55.137269 18588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:55.139534 18595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:25:55.139739 18587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } }
I20260809 06:25:55.140235 18587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:55.145999 18595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:25:55.154588 18595 catalog_manager.cc:1347] Generated new cluster ID: 08e563526049441699341a7c5f9b0abd
I20260809 06:25:55.154721 18595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:25:55.174754 18595 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:25:55.176419 18595 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:25:55.182389 18595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: Generated new TSK 0
I20260809 06:25:55.183506 18595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab.
Entry for principal oryx/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab.
I20260809 06:25:55.256814 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45687
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab
--principal=oryx/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:55.622062 18618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:55.622361 18618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:55.622469 18618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:55.630155 18618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:55.630287 18618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:55.630481 18618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:55.641291 18618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab
--principal=oryx/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:55.642704 18618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:55.644093 18618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:55.657586 18626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:55.657644 18629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:55.658147 18625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:55.658556 18618 server_base.cc:1034] running on GCE node
I20260809 06:25:55.659364 18618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:55.660864 18618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:55.662113 18618 hybrid_clock.cc:648] HybridClock initialized: now 1786256755662041 us; error 71 us; skew 500 ppm
I20260809 06:25:55.662464 18618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:55 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256755, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:55.665835 18618 init.cc:377] Logged in from keytab as oryx/127.8.131.1@KRBTEST.COM (short username oryx)
I20260809 06:25:55.667258 18618 webserver.cc:460] Webserver started at http://127.8.131.1:40165/ using document root <none> and password file <none>
I20260809 06:25:55.668006 18618 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:55.668110 18618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:55.668362 18618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:55.671370 18618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "bb66859753634ddd90ab08678d482835"
format_stamp: "Formatted at 2026-08-09 06:25:55 on dist-test-slave-mjjr"
server_key: "ef76d7509839581939096a081aebecf5"
server_key_iv: "2731871e5fa31e8cd6806c2bdaf17479"
server_key_version: "encryptionkey@0"
I20260809 06:25:55.672134 18618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "bb66859753634ddd90ab08678d482835"
format_stamp: "Formatted at 2026-08-09 06:25:55 on dist-test-slave-mjjr"
server_key: "ef76d7509839581939096a081aebecf5"
server_key_iv: "2731871e5fa31e8cd6806c2bdaf17479"
server_key_version: "encryptionkey@0"
I20260809 06:25:55.677261 18618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:25:55.680595 18638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:55.681969 18618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:55.682137 18618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "bb66859753634ddd90ab08678d482835"
format_stamp: "Formatted at 2026-08-09 06:25:55 on dist-test-slave-mjjr"
server_key: "ef76d7509839581939096a081aebecf5"
server_key_iv: "2731871e5fa31e8cd6806c2bdaf17479"
server_key_version: "encryptionkey@0"
I20260809 06:25:55.682334 18618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:55.695397 18618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:55.699352 18618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:55.699764 18618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:55.700766 18618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:55.702159 18618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:55.702244 18618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:55.702422 18618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:55.702487 18618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:55.736192 18618 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:41977
I20260809 06:25:55.736241 18799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:41977 every 8 connection(s)
I20260809 06:25:55.737524 18618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:55.745918 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 18618
I20260809 06:25:55.746136 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:25:55.746934 8716 external_mini_cluster.cc:1428] Setting key c55cfd7ab21372331323402230c1c6df
Aug 09 06:25:55 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256755, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:25:55.763566 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:55.740163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:46379 (local address 127.8.131.62:45687)
0809 06:25:55.740657 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:55.740662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:55.741635 (+ 973us) server_negotiation.cc:407] Connection header received
0809 06:25:55.743450 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:55.743457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:55.743562 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:55.743710 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:55.745228 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:55.746344 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:55.748273 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:55.748942 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:55.749081 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:55.757041 (+ 7960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:55.757047 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:55.757050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:55.757086 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:55.760017 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:55.761048 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:55.761052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:55.761054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:55.761177 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:55.761735 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:55.761737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:55.761739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:55.762020 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:55.762145 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:55.763089 (+ 944us) server_negotiation.cc:299] Negotiation successful
0809 06:25:55.763297 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":387,"thread_start_us":229,"threads_started":1}
I20260809 06:25:55.765297 18808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:55.740719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.1:46379)
0809 06:25:55.741462 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:55.741501 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:55.743164 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:55.743948 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:55.743958 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:55.744485 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:55.745046 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:55.745062 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:55.746595 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:55.746604 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:55.748125 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:55.748138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:55.749191 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:55.749199 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:55.749501 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:55.753866 (+ 4365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:55.753905 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:55.756782 (+ 2877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:55.760181 (+ 3399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:55.760193 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:55.760215 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:55.760912 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:55.761306 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:55.761313 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:55.761320 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:55.761622 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:55.762163 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:55.762175 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:55.762667 (+ 492us) client_negotiation.cc:769] Sending connection context
0809 06:25:55.763013 (+ 346us) client_negotiation.cc:240] Negotiation successful
0809 06:25:55.763728 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":300,"threads_started":1}
I20260809 06:25:55.767063 18801 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45687
I20260809 06:25:55.767390 18801 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:55.768208 18801 heartbeater.cc:511] Master 127.8.131.62:45687 requested a full tablet report, sending...
I20260809 06:25:55.770496 18532 ts_manager.cc:194] Registered new tserver with Master: bb66859753634ddd90ab08678d482835 (127.8.131.1:41977)
I20260809 06:25:55.772045 18532 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:46379
WARNING: no policy specified for oryx/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab.
Entry for principal oryx/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab.
I20260809 06:25:55.804629 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45687
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab
--principal=oryx/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:56.142946 18812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:56.143235 18812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:56.143334 18812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:56.150768 18812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:56.150892 18812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:56.151024 18812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:56.161325 18812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab
--principal=oryx/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:56.162647 18812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:56.164011 18812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:56.177503 18823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:56.177461 18819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:56.177551 18820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:56.183480 18812 server_base.cc:1034] running on GCE node
I20260809 06:25:56.184270 18812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:56.185242 18812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:56.186440 18812 hybrid_clock.cc:648] HybridClock initialized: now 1786256756186396 us; error 39 us; skew 500 ppm
I20260809 06:25:56.186832 18812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:56 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256756, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:56.190161 18812 init.cc:377] Logged in from keytab as oryx/127.8.131.2@KRBTEST.COM (short username oryx)
I20260809 06:25:56.191706 18812 webserver.cc:460] Webserver started at http://127.8.131.2:37395/ using document root <none> and password file <none>
I20260809 06:25:56.192394 18812 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:56.192499 18812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:56.192749 18812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:56.195585 18812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "b583934b073a4f10a5acf20d959e17b7"
format_stamp: "Formatted at 2026-08-09 06:25:56 on dist-test-slave-mjjr"
server_key: "86948a03456cf222f89816d5e557e884"
server_key_iv: "aee6a2f0616d8d2a2552badce8b26d6f"
server_key_version: "encryptionkey@0"
I20260809 06:25:56.196350 18812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "b583934b073a4f10a5acf20d959e17b7"
format_stamp: "Formatted at 2026-08-09 06:25:56 on dist-test-slave-mjjr"
server_key: "86948a03456cf222f89816d5e557e884"
server_key_iv: "aee6a2f0616d8d2a2552badce8b26d6f"
server_key_version: "encryptionkey@0"
I20260809 06:25:56.201553 18812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:25:56.205031 18835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:56.206370 18812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:56.206593 18812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "b583934b073a4f10a5acf20d959e17b7"
format_stamp: "Formatted at 2026-08-09 06:25:56 on dist-test-slave-mjjr"
server_key: "86948a03456cf222f89816d5e557e884"
server_key_iv: "aee6a2f0616d8d2a2552badce8b26d6f"
server_key_version: "encryptionkey@0"
I20260809 06:25:56.206811 18812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:56.224716 18812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:56.228791 18812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:56.229111 18812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:56.230106 18812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:56.231617 18812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:56.231751 18812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 06:25:56.231909 18812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:56.231986 18812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:56.267474 18812 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:36541
I20260809 06:25:56.267529 19006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:36541 every 8 connection(s)
I20260809 06:25:56.268862 18812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:56.270277 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 18812
I20260809 06:25:56.270439 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:25:56.270864 8716 external_mini_cluster.cc:1428] Setting key acbea0296f46d808d2b23cffcf7dc2ae
Aug 09 06:25:56 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256756, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:25:56.292220 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:56.273233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:59757 (local address 127.8.131.62:45687)
0809 06:25:56.273752 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:56.273760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:56.274925 (+ 1165us) server_negotiation.cc:407] Connection header received
0809 06:25:56.277174 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:56.277182 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:56.277332 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:56.277493 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:56.279065 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:56.279901 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:56.281497 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:56.281916 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:56.282055 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:56.285940 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:56.285956 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:56.285963 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:56.286013 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:56.288820 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:56.289690 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:56.289698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:56.289702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:56.289847 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:56.290336 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:56.290341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:56.290343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:56.290642 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:56.290806 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:56.291553 (+ 747us) server_negotiation.cc:299] Negotiation successful
0809 06:25:56.291822 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":261,"threads_started":1}
I20260809 06:25:56.293269 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:56.273895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.2:59757)
0809 06:25:56.274732 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:56.274788 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:56.276843 (+ 2055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:56.277756 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:56.277765 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:56.278299 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:56.278880 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:56.278897 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:56.280064 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:56.280072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:56.281342 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:56.281354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:56.282166 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:56.282174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:56.282411 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:56.283127 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:56.283154 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:56.285683 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:56.288981 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:56.288991 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:56.289011 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:56.289559 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:56.289970 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:56.289976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:56.289980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:56.290190 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:56.290780 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:56.290789 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:56.291167 (+ 378us) client_negotiation.cc:769] Sending connection context
0809 06:25:56.291473 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 06:25:56.292088 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":363,"threads_started":1}
I20260809 06:25:56.294893 19007 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45687
I20260809 06:25:56.295280 19007 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.8.131.3@KRBTEST.COM; defaulting to no policy
I20260809 06:25:56.296296 19007 heartbeater.cc:511] Master 127.8.131.62:45687 requested a full tablet report, sending...
I20260809 06:25:56.297844 18532 ts_manager.cc:194] Registered new tserver with Master: b583934b073a4f10a5acf20d959e17b7 (127.8.131.2:36541)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.3@KRBTEST.COM" created.
I20260809 06:25:56.298781 18532 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:59757
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab.
Entry for principal oryx/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab.
I20260809 06:25:56.327231 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45687
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab
--principal=oryx/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:56.665652 19016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:56.666090 19016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:56.666251 19016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:56.676363 19016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:56.676494 19016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:56.676612 19016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:25:56.686791 19016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab
--principal=oryx/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:56.688126 19016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:56.689487 19016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:56.703274 19022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:56.703150 19024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:56.703235 19026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:56.704190 19016 server_base.cc:1034] running on GCE node
I20260809 06:25:56.704834 19016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:56.705721 19016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:56.706918 19016 hybrid_clock.cc:648] HybridClock initialized: now 1786256756706846 us; error 54 us; skew 500 ppm
I20260809 06:25:56.707176 19016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:56 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256756, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:56.710480 19016 init.cc:377] Logged in from keytab as oryx/127.8.131.3@KRBTEST.COM (short username oryx)
I20260809 06:25:56.712123 19016 webserver.cc:460] Webserver started at http://127.8.131.3:33633/ using document root <none> and password file <none>
I20260809 06:25:56.712792 19016 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:56.712895 19016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:56.713172 19016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:56.716259 19016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "470fe37812bf43fca27d30d336572ade"
format_stamp: "Formatted at 2026-08-09 06:25:56 on dist-test-slave-mjjr"
server_key: "9b76280d07ec66310e873bf094c82267"
server_key_iv: "750a09ddcfd4c87658f633ef569f6a47"
server_key_version: "encryptionkey@0"
I20260809 06:25:56.717028 19016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "470fe37812bf43fca27d30d336572ade"
format_stamp: "Formatted at 2026-08-09 06:25:56 on dist-test-slave-mjjr"
server_key: "9b76280d07ec66310e873bf094c82267"
server_key_iv: "750a09ddcfd4c87658f633ef569f6a47"
server_key_version: "encryptionkey@0"
I20260809 06:25:56.722594 19016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:25:56.726135 19036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:56.727454 19016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:25:56.727665 19016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "470fe37812bf43fca27d30d336572ade"
format_stamp: "Formatted at 2026-08-09 06:25:56 on dist-test-slave-mjjr"
server_key: "9b76280d07ec66310e873bf094c82267"
server_key_iv: "750a09ddcfd4c87658f633ef569f6a47"
server_key_version: "encryptionkey@0"
I20260809 06:25:56.727883 19016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:56.751345 19016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:56.755448 19016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:56.755837 19016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:56.756826 19016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:56.758249 19016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:56.758355 19016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:56.758512 19016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:56.758585 19016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:56.775313 18801 heartbeater.cc:503] Master 127.8.131.62:45687 was elected leader, sending a full tablet report...
I20260809 06:25:56.792486 19016 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:39623
I20260809 06:25:56.792606 19211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:39623 every 8 connection(s)
I20260809 06:25:56.793859 19016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:25:56.803548 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 19016
I20260809 06:25:56.803800 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:25:56.804536 8716 external_mini_cluster.cc:1428] Setting key b15c02272dc64c1b24ad11dabee2084d
Aug 09 06:25:56 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256756, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:25:56.814555 19217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:56.796487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:52113 (local address 127.8.131.62:45687)
0809 06:25:56.797024 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:56.797029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:56.797725 (+ 696us) server_negotiation.cc:407] Connection header received
0809 06:25:56.799549 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:56.799555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:56.799728 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:56.799857 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:56.801368 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:56.802151 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:56.803699 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:56.804414 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:56.804542 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:56.808426 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:56.808431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:56.808435 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:56.808467 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:56.811306 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:56.812156 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:56.812162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:56.812165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:56.812296 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:56.812723 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:56.812729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:56.812732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:56.813017 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:56.813186 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:56.813906 (+ 720us) server_negotiation.cc:299] Negotiation successful
0809 06:25:56.814150 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":303,"threads_started":1}
I20260809 06:25:56.815538 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:56.796863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.3:52113)
0809 06:25:56.797555 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:56.797597 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:56.799274 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:56.800111 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:56.800119 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:56.800631 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:56.801185 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:56.801202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:56.802314 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:56.802319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:56.803531 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:56.803543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:56.804858 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:56.804865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:56.805144 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:56.805866 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:56.805888 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:56.808208 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:56.811466 (+ 3258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:56.811474 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:56.811492 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:56.812024 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:56.812408 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:56.812412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:56.812416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:56.812607 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:56.813157 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:56.813166 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:56.813523 (+ 357us) client_negotiation.cc:769] Sending connection context
0809 06:25:56.813814 (+ 291us) client_negotiation.cc:240] Negotiation successful
0809 06:25:56.814390 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":272,"threads_started":1}
I20260809 06:25:56.817271 19214 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45687
I20260809 06:25:56.817674 19214 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:56.818531 19214 heartbeater.cc:511] Master 127.8.131.62:45687 requested a full tablet report, sending...
I20260809 06:25:56.820209 18532 ts_manager.cc:194] Registered new tserver with Master: 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623)
I20260809 06:25:56.821043 18532 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:52113
I20260809 06:25:56.832888 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.131.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.61.keytab.
Entry for principal oryx/127.8.131.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.61.keytab.
I20260809 06:25:56.887356 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.61:33805
--webserver_interface=127.8.131.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.61.keytab
--principal=oryx/127.8.131.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:45687,127.8.131.61:33805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:57.229214 19223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:57.229504 19223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:57.229597 19223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:57.237211 19223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:57.237327 19223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:57.237387 19223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:57.237447 19223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:57.237490 19223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:57.247547 19223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:45687,127.8.131.61:33805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.61.keytab
--principal=oryx/127.8.131.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.61:33805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.131.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
W20260809 06:25:57.247918 19223 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.131.62:45687,127.8.131.61:33805'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 06:25:57.248939 19223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:57.250288 19223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:57.262128 19229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:57.262096 19233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:57.262197 19230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:57.268527 19223 server_base.cc:1034] running on GCE node
I20260809 06:25:57.269433 19223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:57.270912 19223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:57.272162 19223 hybrid_clock.cc:648] HybridClock initialized: now 1786256757272094 us; error 43 us; skew 500 ppm
I20260809 06:25:57.272514 19223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:57 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256757, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:57.275753 19223 init.cc:377] Logged in from keytab as oryx/127.8.131.61@KRBTEST.COM (short username oryx)
I20260809 06:25:57.277346 19223 webserver.cc:460] Webserver started at http://127.8.131.61:33629/ using document root <none> and password file <none>
I20260809 06:25:57.278079 19223 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:57.278211 19223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:57.278545 19223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:57.281646 19223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data/instance:
uuid: "52b995272cdd470b9dd9af9cffbc4879"
format_stamp: "Formatted at 2026-08-09 06:25:57 on dist-test-slave-mjjr"
server_key: "78ce01c2cbe91fed8a8471f142a4241d"
server_key_iv: "9356b03da950805eb3635db3c2a2b2a6"
server_key_version: "encryptionkey@0"
I20260809 06:25:57.282452 19223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal/instance:
uuid: "52b995272cdd470b9dd9af9cffbc4879"
format_stamp: "Formatted at 2026-08-09 06:25:57 on dist-test-slave-mjjr"
server_key: "78ce01c2cbe91fed8a8471f142a4241d"
server_key_iv: "9356b03da950805eb3635db3c2a2b2a6"
server_key_version: "encryptionkey@0"
I20260809 06:25:57.287808 19223 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 06:25:57.291214 19243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:57.292582 19223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:57.292768 19223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal
uuid: "52b995272cdd470b9dd9af9cffbc4879"
format_stamp: "Formatted at 2026-08-09 06:25:57 on dist-test-slave-mjjr"
server_key: "78ce01c2cbe91fed8a8471f142a4241d"
server_key_iv: "9356b03da950805eb3635db3c2a2b2a6"
server_key_version: "encryptionkey@0"
I20260809 06:25:57.292994 19223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:57.301611 19007 heartbeater.cc:503] Master 127.8.131.62:45687 was elected leader, sending a full tablet report...
I20260809 06:25:57.327921 19223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:57.332338 19223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:57.332659 19223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:57.351950 19321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.61:33805 every 8 connection(s)
I20260809 06:25:57.352010 19223 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.61:33805
I20260809 06:25:57.353477 19223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
I20260809 06:25:57.357175 19322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:57.362530 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 19223
I20260809 06:25:57.363759 19322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:25:57 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256757, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.61@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
Aug 09 06:25:57 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.61@KRBTEST.COM
I20260809 06:25:57.385314 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.366855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60830 (local address 127.8.131.62:45687)
0809 06:25:57.368322 (+ 1467us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:57.368328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:57.368349 (+ 21us) server_negotiation.cc:407] Connection header received
0809 06:25:57.368559 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:57.368565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:57.368646 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:57.368780 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:57.370494 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.371743 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.373856 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.374426 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.374560 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.378725 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:57.378730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:57.378733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:57.378770 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:57.381798 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.382615 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.382621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.382624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.382748 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.383175 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.383180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.383184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.383540 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:57.383717 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:57.384510 (+ 793us) server_negotiation.cc:299] Negotiation successful
0809 06:25:57.384742 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1346,"thread_start_us":1214,"threads_started":1}
I20260809 06:25:57.386255 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.366772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:60830)
0809 06:25:57.367546 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:57.367594 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:57.368318 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:57.368868 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:57.368874 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:57.369751 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:57.370308 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.370332 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.371877 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.371881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.373713 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.373728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.374644 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.374649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.374832 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.375546 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:57.375598 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:57.378534 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:57.381942 (+ 3408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.381962 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.381977 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.382486 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.382859 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.382863 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.382867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.383061 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.383695 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:57.383704 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:57.384139 (+ 435us) client_negotiation.cc:769] Sending connection context
0809 06:25:57.384416 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 06:25:57.384937 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":53,"spinlock_wait_cycles":15104,"thread_start_us":339,"threads_started":1}
I20260809 06:25:57.386273 19326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.364861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44066 (local address 127.8.131.61:33805)
0809 06:25:57.365666 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:57.365679 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:57.365729 (+ 50us) server_negotiation.cc:407] Connection header received
0809 06:25:57.368306 (+ 2577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:57.368332 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:57.368648 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:57.368837 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:57.370716 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.372383 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.374102 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.374880 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.374976 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.379482 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:57.379490 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:57.379502 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:57.379531 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:57.382270 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.382955 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.382962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.382964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.383096 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.383498 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.383506 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.383509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.384106 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:57.384429 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:57.384774 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 06:25:57.385140 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":567,"thread_start_us":336,"threads_started":1}
I20260809 06:25:57.389142 19322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:25:57 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256757, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.61@KRBTEST.COM for oryx/127.8.131.61@KRBTEST.COM
I20260809 06:25:57.400924 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.389881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.0.0.1:44074)
0809 06:25:57.390060 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:57.390078 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:57.390176 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:57.390484 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:57.390487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:57.390793 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:57.390926 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.390933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.391908 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.391912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.392736 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.392746 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.393390 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.393394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.393529 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.394109 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:57.394127 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:57.396104 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:57.398826 (+ 2722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.398829 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.398832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.399187 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.399575 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.399578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.399580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.399769 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.400246 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:57.400249 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:57.400369 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 06:25:57.400482 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 06:25:57.400687 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 06:25:57.401116 19326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.390125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44074 (local address 127.8.131.61:33805)
0809 06:25:57.390257 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:57.390261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:57.390275 (+ 14us) server_negotiation.cc:407] Connection header received
0809 06:25:57.390311 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:57.390314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:57.390364 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:57.390445 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:57.391048 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.391786 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.392852 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.393271 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.393363 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.396294 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:57.396298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:57.396301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:57.396328 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:57.398708 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.399305 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.399311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.399314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.399464 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.399859 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.399863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.399865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.400140 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:57.400246 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:57.400569 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 06:25:57.400819 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":47}
I20260809 06:25:57.405812 19322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Bootstrap starting.
I20260809 06:25:57.408942 19322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:57.410005 19322 log.cc:824] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:57.412685 19322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: No bootstrap required, opened a new log
I20260809 06:25:57.419497 19322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } }
I20260809 06:25:57.419953 19322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:57.420063 19322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b995272cdd470b9dd9af9cffbc4879, State: Initialized, Role: FOLLOWER
I20260809 06:25:57.420694 19322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } }
I20260809 06:25:57.421518 19335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } } }
I20260809 06:25:57.421808 19335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:25:57.422537 19322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:25:57.431269 19350 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:25:57.431421 19350 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:25:57.433779 19223 master_runner.cc:428] Detected that this master 52b995272cdd470b9dd9af9cffbc4879 is joining an existing cluster
I20260809 06:25:57.433915 19223 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.131.61:33805
I20260809 06:25:57.457202 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.437784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60834 (local address 127.8.131.62:45687)
0809 06:25:57.437971 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:57.437978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:57.438663 (+ 685us) server_negotiation.cc:407] Connection header received
0809 06:25:57.438906 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:57.438912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:57.439012 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:57.439124 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:57.441228 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.442769 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.445571 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.446273 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.446379 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.449147 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:57.449153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:57.449158 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:57.449206 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:57.453018 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.453917 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.453923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.453926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.454123 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.454615 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.454623 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.454628 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.455322 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:57.455492 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:57.456545 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0809 06:25:57.456829 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 06:25:57.458334 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.437780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:60834)
0809 06:25:57.438523 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:57.438547 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:57.438730 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:57.439179 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:57.439186 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:57.440389 (+ 1203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:57.440989 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.441009 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.443043 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.443052 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.445360 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.445381 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.446752 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.446763 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.446975 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.447752 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:57.447777 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:57.448919 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:57.453198 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.453204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.453208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.453748 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.454258 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.454262 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.454265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.454455 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.455561 (+ 1106us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:57.455566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:57.455850 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 06:25:57.457426 (+ 1576us) client_negotiation.cc:240] Negotiation successful
0809 06:25:57.457833 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":392,"threads_started":1}
I20260809 06:25:57.463367 19326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.437987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.8.131.61:33805)
0809 06:25:57.438215 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:57.438221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:57.445715 (+ 7494us) server_negotiation.cc:407] Connection header received
0809 06:25:57.446102 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:57.446107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:57.446179 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:57.446250 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:57.447787 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.448920 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.450925 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.451807 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.451925 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.454362 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:57.454368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:57.454372 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:57.454403 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:57.458160 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.460915 (+ 2755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.460922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.460928 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.461132 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.461586 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.461590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.461592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.462012 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:57.462183 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:57.462691 (+ 508us) server_negotiation.cc:299] Negotiation successful
0809 06:25:57.462929 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 06:25:57.466697 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.438374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.0.0.1:44078)
0809 06:25:57.445629 (+ 7255us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:57.445648 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:57.445974 (+ 326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:57.446436 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:57.446440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:57.447222 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:57.447595 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.447614 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.449143 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.449149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.450773 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.450789 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.452241 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.452250 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.452435 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.453086 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:57.453111 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:57.454166 (+ 1055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:57.460174 (+ 6008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.460181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.460188 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.460785 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.461289 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.461297 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.461304 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.461479 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.462253 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:57.462258 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:57.462443 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 06:25:57.465352 (+ 2909us) client_negotiation.cc:240] Negotiation successful
0809 06:25:57.465723 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7122,"thread_start_us":7342,"threads_started":1}
WARNING: no policy specified for oryx/127.8.131.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.60@KRBTEST.COM" created.
Aug 09 06:25:57 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256755, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.62@KRBTEST.COM for oryx/127.8.131.61@KRBTEST.COM
I20260809 06:25:57.494833 19326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.466399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44088 (local address 127.8.131.61:33805)
0809 06:25:57.466602 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:57.466614 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:57.475731 (+ 9117us) server_negotiation.cc:407] Connection header received
0809 06:25:57.476310 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:57.476314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:57.476410 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:57.476537 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:57.477971 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.479077 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.481090 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.481920 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.482066 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.486966 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:57.486973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:57.486979 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:57.487024 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:57.491003 (+ 3979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.492150 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.492156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.492160 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.492339 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.492959 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.492964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.492968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.493364 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:57.493529 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:57.494171 (+ 642us) server_negotiation.cc:299] Negotiation successful
0809 06:25:57.494462 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 06:25:57.495020 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.465850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.0.0.1:44088)
0809 06:25:57.475537 (+ 9687us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:57.475576 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:57.475848 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:57.476604 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:57.476612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:57.477438 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:57.477761 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.477810 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.479244 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.479250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.480918 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.480938 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.482114 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.482121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.482346 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.483287 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:57.483314 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:57.486716 (+ 3402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:57.491204 (+ 4488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.491223 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.491235 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.491992 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.492491 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.492518 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.492525 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.492748 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.493539 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:57.493550 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:57.493791 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 06:25:57.494021 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 06:25:57.494448 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9453,"thread_start_us":9298,"threads_started":1}
I20260809 06:25:57.497990 18532 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.131.61:33805: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879"
member_type: NON_VOTER
last_known_addr {
host: "127.8.131.61"
port: 33805
}
attrs {
promote: true
}
}
dest_uuid: "467cc2d070a8428d9cd4281d718a7875"
cas_config_opid_index: -1
I20260809 06:25:57.499362 18532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } }
I20260809 06:25:57.505666 19372 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 52b995272cdd470b9dd9af9cffbc4879 (127.8.131.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } } }
I20260809 06:25:57.509743 19372 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.131.61:33805
I20260809 06:25:57.510354 19280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:57.510922 19223 master.cc:537] Master@127.8.131.61:33805 shutting down...
I20260809 06:25:57.510975 19372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } } }
I20260809 06:25:57.511590 19372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.60.keytab.
Entry for principal oryx/127.8.131.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.60.keytab.
I20260809 06:25:57.514299 19280 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Refusing update from remote peer 467cc2d070a8428d9cd4281d718a7875: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 06:25:57.515441 19372 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:25:57.516896 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.60:39643
--webserver_interface=127.8.131.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.60.keytab
--principal=oryx/127.8.131.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:45687,127.8.131.61:33805,127.8.131.60:39643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:25:57.518801 18512 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 06:25:57.531555 19223 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:57.532119 19223 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:57.532397 19223 tablet_replica.cc:330] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: stopping tablet replica
W20260809 06:25:57.533423 18512 consensus_peers.cc:487] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 -> Peer 52b995272cdd470b9dd9af9cffbc4879 (127.8.131.61:33805): Couldn't send request to peer 52b995272cdd470b9dd9af9cffbc4879. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 06:25:57.540468 19223 master.cc:552] Master@127.8.131.61:33805 shutdown complete.
I20260809 06:25:57.540663 19223 master_runner.cc:305] Clearing existing system tablet
I20260809 06:25:57.541254 19223 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:57.541366 19223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:57.544932 19223 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 06:25:57.546828 19380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:57.547178 19223 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:57.547503 19223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 06:25:57.547602 19223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal
uuid: "52b995272cdd470b9dd9af9cffbc4879"
format_stamp: "Formatted at 2026-08-09 06:25:57 on dist-test-slave-mjjr"
server_key: "78ce01c2cbe91fed8a8471f142a4241d"
server_key_iv: "9356b03da950805eb3635db3c2a2b2a6"
server_key_version: "encryptionkey@0"
I20260809 06:25:57.553506 19223 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:57.560567 19223 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 06:25:57.560756 19223 log.cc:1196] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 06:25:57.561194 19223 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Deleting consensus metadata
I20260809 06:25:57.561467 19223 master_runner.cc:315] Copying system tablet from 127.8.131.62:45687
I20260809 06:25:57.564402 19223 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: tablet copy: Beginning tablet copy session from remote peer at address 127.8.131.62:45687
I20260809 06:25:57.580542 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.565282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60848 (local address 127.8.131.62:45687)
0809 06:25:57.565482 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:57.565488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:57.566031 (+ 543us) server_negotiation.cc:407] Connection header received
0809 06:25:57.566223 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:57.566229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:57.566332 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:57.566444 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:57.567486 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.568680 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.570389 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.571088 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.571218 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.573588 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:57.573598 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:57.573602 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:57.573645 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:57.577293 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.578144 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.578150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.578154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.578304 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:57.578793 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:57.578798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:57.578801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:57.579189 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:57.579346 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:57.579910 (+ 564us) server_negotiation.cc:299] Negotiation successful
0809 06:25:57.580154 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:25:57.580626 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.565206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:60848)
0809 06:25:57.565887 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:57.565917 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:57.566080 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:57.566513 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:57.566517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:57.567033 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:57.567309 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.567321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.568861 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.568867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.570234 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.570249 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.571256 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.571262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.571446 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.572200 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:57.572222 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:57.573338 (+ 1116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:57.577456 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.577462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.577465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.578012 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.578451 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:57.578459 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:57.578464 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:57.578670 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:57.579330 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:57.579335 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:57.579549 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 06:25:57.579781 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 06:25:57.580137 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":393,"threads_started":1}
I20260809 06:25:57.581938 18561 tablet_copy_service.cc:140] P 467cc2d070a8428d9cd4281d718a7875: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 52b995272cdd470b9dd9af9cffbc4879 ({username='oryx', principal='oryx/127.8.131.61@KRBTEST.COM'} at 127.0.0.1:60848)
I20260809 06:25:57.582222 18561 tablet_copy_service.cc:161] P 467cc2d070a8428d9cd4281d718a7875: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 52b995272cdd470b9dd9af9cffbc4879 at {username='oryx', principal='oryx/127.8.131.61@KRBTEST.COM'} at 127.0.0.1:60848: session id = 52b995272cdd470b9dd9af9cffbc4879-00000000000000000000000000000000
I20260809 06:25:57.586930 18561 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 06:25:57.589628 19223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 06:25:57.597086 19223 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: tablet copy: Starting download of 0 data blocks...
I20260809 06:25:57.597445 19223 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: tablet copy: Starting download of 1 WAL segments...
I20260809 06:25:57.600548 19223 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 06:25:57.605566 18561 tablet_copy_service.cc:342] P 467cc2d070a8428d9cd4281d718a7875: Request end of tablet copy session 52b995272cdd470b9dd9af9cffbc4879-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.131.61@KRBTEST.COM'} at 127.0.0.1:60848
I20260809 06:25:57.605834 18561 tablet_copy_service.cc:433] P 467cc2d070a8428d9cd4281d718a7875: ending tablet copy session 52b995272cdd470b9dd9af9cffbc4879-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 52b995272cdd470b9dd9af9cffbc4879
I20260809 06:25:57.618263 19223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:57.621371 19391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:57.623796 19393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:57.625469 19223 server_base.cc:1034] running on GCE node
W20260809 06:25:57.625707 19396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:57.626217 19223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:57.626915 19223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:57.628211 19223 hybrid_clock.cc:648] HybridClock initialized: now 1786256757628149 us; error 28 us; skew 500 ppm
I20260809 06:25:57.628451 19223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:57 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256757, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:57.630386 19223 init.cc:377] Logged in from keytab as oryx/127.8.131.61@KRBTEST.COM (short username oryx)
I20260809 06:25:57.631843 19223 webserver.cc:460] Webserver started at http://127.8.131.61:35555/ using document root <none> and password file <none>
I20260809 06:25:57.632215 19223 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:57.632320 19223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:57.634922 19223 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 06:25:57.636864 19408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:57.637328 19223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 06:25:57.637501 19223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal
uuid: "52b995272cdd470b9dd9af9cffbc4879"
format_stamp: "Formatted at 2026-08-09 06:25:57 on dist-test-slave-mjjr"
server_key: "78ce01c2cbe91fed8a8471f142a4241d"
server_key_iv: "9356b03da950805eb3635db3c2a2b2a6"
server_key_version: "encryptionkey@0"
I20260809 06:25:57.637672 19223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:57.646431 19223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:57.649986 19223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:57.690706 19223 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.61:33805
I20260809 06:25:57.690747 19484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.61:33805 every 8 connection(s)
I20260809 06:25:57.691823 19223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
I20260809 06:25:57.699153 19486 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:25:57.706199 19486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Bootstrap starting.
I20260809 06:25:57.719097 19486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:25:57.719570 19486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Bootstrap complete.
I20260809 06:25:57.720418 19486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } }
I20260809 06:25:57.720651 19486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52b995272cdd470b9dd9af9cffbc4879, State: Initialized, Role: LEARNER
I20260809 06:25:57.720958 19486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } }
I20260809 06:25:57.721778 19495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } } }
I20260809 06:25:57.722105 19486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:57.722223 19495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: This master's current role is: LEARNER
I20260809 06:25:57.732345 19516 catalog_manager.cc:1259] Loaded cluster ID: 08e563526049441699341a7c5f9b0abd
I20260809 06:25:57.732478 19516 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: loading cluster ID for follower catalog manager: success
I20260809 06:25:57.737147 19516 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: acquiring CA information for follower catalog manager: success
I20260809 06:25:57.740882 19516 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:25:57.823693 19214 heartbeater.cc:503] Master 127.8.131.62:45687 was elected leader, sending a full tablet report...
W20260809 06:25:57.974128 19378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:57.974459 19378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:57.974635 19378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:57.982389 19378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:25:57.982540 19378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:57.982618 19378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:57.982725 19378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:25:57.982791 19378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:25:57.987534 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.980987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.0.0.1:44098)
0809 06:25:57.981199 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:57.981217 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:57.981322 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:57.982148 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:57.982154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:57.982186 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:57.982382 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.982391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.983926 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.983932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.985288 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:57.985300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.986603 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.986609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:57.986918 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.986967 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 06:25:57.987080 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 06:25:57.987226 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 06:25:57.987562 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:57.981191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44098 (local address 127.8.131.61:33805)
0809 06:25:57.981792 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:57.981797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:57.981813 (+ 16us) server_negotiation.cc:407] Connection header received
0809 06:25:57.981873 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:57.981877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:57.981959 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:57.982096 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:57.982535 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.983771 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.985450 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:57.986406 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:57.986582 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:57.986846 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:57.987110 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 06:25:57.987243 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":356,"threads_started":1}
I20260809 06:25:57.988626 19448 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 06:25:57.989184 19495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 467cc2d070a8428d9cd4281d718a7875. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } } }
I20260809 06:25:57.989481 19495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: This master's current role is: LEARNER
I20260809 06:25:57.993507 19378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:45687,127.8.131.61:33805,127.8.131.60:39643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.60.keytab
--principal=oryx/127.8.131.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.60:39643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.131.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:57.994881 19378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:57.996222 19378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:58.008104 19527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:58.008136 19534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:58.008162 19528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:58.009187 19378 server_base.cc:1034] running on GCE node
I20260809 06:25:58.009989 19378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:58.011675 19378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:58.012909 19378 hybrid_clock.cc:648] HybridClock initialized: now 1786256758012865 us; error 63 us; skew 500 ppm
I20260809 06:25:58.013310 19378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256758, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:58.016673 19378 init.cc:377] Logged in from keytab as oryx/127.8.131.60@KRBTEST.COM (short username oryx)
I20260809 06:25:58.018081 19378 webserver.cc:460] Webserver started at http://127.8.131.60:37213/ using document root <none> and password file <none>
I20260809 06:25:58.018762 19378 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:58.018913 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:58.019229 19378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:25:58.022375 19378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data/instance:
uuid: "2e204de3ce804cdaa3f588fa4a03e4aa"
format_stamp: "Formatted at 2026-08-09 06:25:58 on dist-test-slave-mjjr"
server_key: "9af6ee429c7cd560066ab63f162d95a4"
server_key_iv: "4f6990187b699685d9df1aae11e90380"
server_key_version: "encryptionkey@0"
I20260809 06:25:58.023237 19378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal/instance:
uuid: "2e204de3ce804cdaa3f588fa4a03e4aa"
format_stamp: "Formatted at 2026-08-09 06:25:58 on dist-test-slave-mjjr"
server_key: "9af6ee429c7cd560066ab63f162d95a4"
server_key_iv: "4f6990187b699685d9df1aae11e90380"
server_key_version: "encryptionkey@0"
I20260809 06:25:58.028813 19378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 06:25:58.032461 19545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:58.033842 19378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:58.034052 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal
uuid: "2e204de3ce804cdaa3f588fa4a03e4aa"
format_stamp: "Formatted at 2026-08-09 06:25:58 on dist-test-slave-mjjr"
server_key: "9af6ee429c7cd560066ab63f162d95a4"
server_key_iv: "4f6990187b699685d9df1aae11e90380"
server_key_version: "encryptionkey@0"
I20260809 06:25:58.034262 19378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:58.049857 19378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:58.054061 19378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:58.054351 19378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:58.073661 19378 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.60:39643
I20260809 06:25:58.073750 19619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.60:39643 every 8 connection(s)
I20260809 06:25:58.075189 19378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
I20260809 06:25:58.076283 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 19378
I20260809 06:25:58.079499 19621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:25:58.087935 19621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.60@KRBTEST.COM
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256758, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.60@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:25:58.103091 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.078568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57696 (local address 127.8.131.60:39643)
0809 06:25:58.079396 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.079408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.079450 (+ 42us) server_negotiation.cc:407] Connection header received
0809 06:25:58.081133 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.081153 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.081618 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.081967 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.083938 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.085289 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.087245 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.088370 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.088580 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.094181 (+ 5601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.094192 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.094205 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.094249 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.098298 (+ 4049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.099122 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.099131 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.099139 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.099358 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.099854 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.099859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.099863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.100545 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.100848 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.101367 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.101772 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":391,"threads_started":1}
I20260809 06:25:58.107690 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.090977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60854 (local address 127.8.131.62:45687)
0809 06:25:58.091540 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.091547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.092319 (+ 772us) server_negotiation.cc:407] Connection header received
0809 06:25:58.092598 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.092604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.092692 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.092801 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.094324 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.095320 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.096857 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.097299 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.097463 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.101265 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.101271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.101278 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.101322 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.104601 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.105516 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.105522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.105525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.105701 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.106114 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.106119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.106122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.106440 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.106617 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.107058 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.107267 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":321,"threads_started":1}
I20260809 06:25:58.107712 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.090846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:60854)
0809 06:25:58.092183 (+ 1337us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.092234 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.092449 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.092903 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.092913 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.093774 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.094169 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.094185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.095477 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.095483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.096706 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.096719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.097507 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.097513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.097696 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.098322 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.098339 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.101066 (+ 2727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.104741 (+ 3675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.104751 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.104758 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.105367 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.105828 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.105832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.105834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.105981 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.106586 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.106593 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.106759 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.106938 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.107264 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1057,"thread_start_us":892,"threads_started":1}
I20260809 06:25:58.109364 19621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256758, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.60@KRBTEST.COM for oryx/127.8.131.61@KRBTEST.COM
I20260809 06:25:58.121183 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.110100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.0.0.1:44114)
0809 06:25:58.110317 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.110332 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.110432 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.110770 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.110773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.111080 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.111208 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.111215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.112257 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.112261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.113195 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.113206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.113830 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.113834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.113972 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.114538 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.114555 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.116495 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.119207 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.119211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.119213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.119576 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.119954 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.119957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.119960 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.120087 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.120532 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.120535 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.120649 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.120756 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.120969 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 06:25:58.121233 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.110093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.8.131.61:33805)
0809 06:25:58.110312 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.110318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.110397 (+ 79us) server_negotiation.cc:407] Connection header received
0809 06:25:58.110541 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.110545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.110645 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.110737 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.111339 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.112135 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.113313 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.113710 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.113848 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.116691 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.116696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.116699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.116728 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.119090 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.119723 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.119727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.119730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.119858 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.120177 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.120181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.120182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.120428 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.120546 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.120838 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.121004 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 06:25:58.122299 19621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256758, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.60@KRBTEST.COM for oryx/127.8.131.60@KRBTEST.COM
I20260809 06:25:58.134508 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.122975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:39643 (local address 127.0.0.1:57706)
0809 06:25:58.123269 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.123290 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.123393 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.123712 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.123716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.124084 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.124212 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.124220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.125305 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.125310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.126203 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.126215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.126916 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.126923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.127115 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.127582 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.127598 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.129537 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.132430 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.132434 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.132436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.132864 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.133186 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.133189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.133191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.133333 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.133759 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.133762 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.133888 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.133996 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.134172 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260809 06:25:58.134717 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.122977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.8.131.60:39643)
0809 06:25:58.123140 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.123147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.123340 (+ 193us) server_negotiation.cc:407] Connection header received
0809 06:25:58.123530 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.123533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.123583 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.123699 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.124376 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.125169 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.126338 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.126751 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.126877 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.129698 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.129703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.129706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.129732 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.132320 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.132961 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.132965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.132967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.133082 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.133429 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.133432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.133434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.133665 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.133770 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.134076 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.134399 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":54}
I20260809 06:25:58.139613 19621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: Bootstrap starting.
I20260809 06:25:58.142983 19621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: Neither blocks nor log segments found. Creating new log.
I20260809 06:25:58.144088 19621 log.cc:824] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: Log is configured to *not* fsync() on all Append() calls
I20260809 06:25:58.146580 19621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: No bootstrap required, opened a new log
I20260809 06:25:58.153576 19621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } }
I20260809 06:25:58.153920 19621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:25:58.154059 19621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e204de3ce804cdaa3f588fa4a03e4aa, State: Initialized, Role: FOLLOWER
I20260809 06:25:58.154682 19621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } }
I20260809 06:25:58.155551 19630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } } }
I20260809 06:25:58.155951 19630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:25:58.156623 19621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:25:58.165362 19644 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:25:58.165516 19644 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:25:58.169909 19378 master_runner.cc:428] Detected that this master 2e204de3ce804cdaa3f588fa4a03e4aa is joining an existing cluster
I20260809 06:25:58.170035 19378 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.131.60:39643
I20260809 06:25:58.193426 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.175110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44130 (local address 127.8.131.61:33805)
0809 06:25:58.175304 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.175309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.176297 (+ 988us) server_negotiation.cc:407] Connection header received
0809 06:25:58.176611 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.176618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.176704 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.176780 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.178663 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.179789 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.182063 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.182756 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.182860 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.186055 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.186061 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.186065 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.186102 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.190130 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.190929 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.190934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.190938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.191097 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.191633 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.191660 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.191663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.192037 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.192158 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.192808 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.193075 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 06:25:58.193648 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.175023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57720 (local address 127.8.131.60:39643)
0809 06:25:58.175224 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.175230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.176606 (+ 1376us) server_negotiation.cc:407] Connection header received
0809 06:25:58.176784 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.176789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.176851 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.176916 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.178652 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.179817 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.182051 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.182778 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.182859 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.186049 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.186059 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.186063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.186101 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.190422 (+ 4321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.191249 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.191253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.191256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.191464 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.191969 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.191974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.191976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.192362 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.192521 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.193044 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.193273 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 06:25:58.194679 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.174640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:39643 (local address 127.0.0.1:57720)
0809 06:25:58.176524 (+ 1884us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.176543 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.176675 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.177110 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.177115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.177896 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.178526 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.178536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.180024 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.180029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.181900 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.181915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.183171 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.183182 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.183378 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.184201 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.184267 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.185853 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.190600 (+ 4747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.190604 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.190608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.191114 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.191620 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.191624 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.191627 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.191829 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.192613 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.192621 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.192757 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.193875 (+ 1118us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.194189 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1756,"mutex_wait_us":179,"thread_start_us":1567,"threads_started":1}
I20260809 06:25:58.196276 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.174564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.0.0.1:44130)
0809 06:25:58.176127 (+ 1563us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.176152 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.176425 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.176991 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.176996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.177896 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.178481 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.178503 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.180024 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.180029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.181902 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.181915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.183198 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.183204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.183429 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.184304 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.184322 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.185854 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.190290 (+ 4436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.190296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.190300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.190793 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.191256 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.191260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.191263 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.191499 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.192252 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.192257 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.192505 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.194183 (+ 1678us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.194497 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1407,"mutex_wait_us":157,"spinlock_wait_cycles":61312,"thread_start_us":13246,"threads_started":1}
I20260809 06:25:58.203586 19654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.175023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:60858)
0809 06:25:58.188128 (+ 13105us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.188143 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.188293 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.188812 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.188816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.189659 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.190009 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.190024 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.191541 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.191547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.193273 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.193288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.194526 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.194537 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.194720 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.195370 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.195415 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.196591 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.200817 (+ 4226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.200821 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.200824 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.201317 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.201840 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.201847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.201853 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.202065 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.202740 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.202744 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.202904 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.203026 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.203251 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12961,"thread_start_us":754,"threads_started":1}
I20260809 06:25:58.203896 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.174393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.8.131.62:45687)
0809 06:25:58.174593 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.174599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.188249 (+ 13650us) server_negotiation.cc:407] Connection header received
0809 06:25:58.188424 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.188430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.188531 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.188630 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.190196 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.191331 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.193456 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.194108 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.194206 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.197073 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.197080 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.197085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.197128 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.200683 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.201471 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.201477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.201496 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.201675 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.202209 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.202216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.202219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.202610 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.202782 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.203203 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.203486 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 06:25:58.209475 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 18618
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256755, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.62@KRBTEST.COM for oryx/127.8.131.60@KRBTEST.COM
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.1@KRBTEST.COM
I20260809 06:25:58.223500 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.209148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.8.131.60:39643)
0809 06:25:58.209317 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.209323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.209347 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:25:58.209445 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.209450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.209570 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.209675 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.210584 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.211836 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.213283 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.213872 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.214012 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.217685 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.217691 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.217697 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.217733 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.220728 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.221386 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.221393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.221397 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.221524 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.221935 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.221940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.221951 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.222296 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.222459 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.222936 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.223176 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:25:58.223470 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.208952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:39643 (local address 127.0.0.1:57732)
0809 06:25:58.209141 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.209160 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.209270 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.209724 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.209730 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.210196 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.210396 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.210409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.212006 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.212011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.213123 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.213136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.214019 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.214025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.214203 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.214833 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.214856 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.217466 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.220877 (+ 3411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.220883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.220887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.221258 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.221644 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.221649 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.221653 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.221825 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.222466 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.222474 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.222625 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.222779 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.223063 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 06:25:58.225507 18532 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.131.60:39643: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa"
member_type: NON_VOTER
last_known_addr {
host: "127.8.131.60"
port: 39643
}
attrs {
promote: true
}
}
dest_uuid: "467cc2d070a8428d9cd4281d718a7875"
cas_config_opid_index: 5
I20260809 06:25:58.226451 18532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } }
I20260809 06:25:58.228665 19372 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2e204de3ce804cdaa3f588fa4a03e4aa (127.8.131.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.232658 19448 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 LEARNER]: Refusing update from remote peer 467cc2d070a8428d9cd4281d718a7875: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 06:25:58.233191 19656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.210817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.8.131.1:41977)
0809 06:25:58.211587 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.211596 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.211623 (+ 27us) server_negotiation.cc:407] Connection header received
0809 06:25:58.211729 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.211737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.212024 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.212301 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.213963 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.215050 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.216446 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.217327 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.217485 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.223365 (+ 5880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.223395 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.223415 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.223464 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.227816 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.228730 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.228739 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.228743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.228907 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.229360 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.229367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.229371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.229829 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.231327 (+ 1498us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.231512 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.231942 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":400,"threads_started":1}
I20260809 06:25:58.233948 19587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:25:58.234665 18794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:58.243690 19587 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 FOLLOWER]: Refusing update from remote peer 467cc2d070a8428d9cd4281d718a7875: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 06:25:58.251515 19372 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.131.60:39643
I20260809 06:25:58.251598 19665 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 06:25:58.252295 19378 master.cc:537] Master@127.8.131.60:39643 shutting down...
I20260809 06:25:58.264199 19660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.264596 19660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:58.263727 19658 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 06:25:58.268008 19448 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2e204de3ce804cdaa3f588fa4a03e4aa (127.8.131.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.272626 19495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: NON_VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: true } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.272914 19495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: This master's current role is: LEARNER
I20260809 06:25:58.274070 19660 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: attempting to promote NON_VOTER 52b995272cdd470b9dd9af9cffbc4879 to VOTER
I20260809 06:25:58.274775 19660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } }
I20260809 06:25:58.276777 19448 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 LEARNER]: Refusing update from remote peer 467cc2d070a8428d9cd4281d718a7875: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 06:25:58.277465 19660 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 06:25:58.280010 19665 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 52b995272cdd470b9dd9af9cffbc4879 (127.8.131.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.280721 19448 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 52b995272cdd470b9dd9af9cffbc4879 (127.8.131.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.284127 19665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.284580 19665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:58.284420 19660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.285207 19660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:58.286551 19495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.286803 19495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 06:25:58.287247 18511 consensus_peers.cc:487] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 -> Peer 2e204de3ce804cdaa3f588fa4a03e4aa (127.8.131.60:39643): Couldn't send request to peer 2e204de3ce804cdaa3f588fa4a03e4aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 06:25:58.288342 19378 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:25:58.288767 19378 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:25:58.288973 19378 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: stopping tablet replica
I20260809 06:25:58.295753 19378 master.cc:552] Master@127.8.131.60:39643 shutdown complete.
I20260809 06:25:58.295890 19378 master_runner.cc:305] Clearing existing system tablet
I20260809 06:25:58.296379 19378 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:58.296562 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:58.299535 19378 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 06:25:58.301318 19672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:58.301714 19378 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:58.302139 19378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 06:25:58.302290 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal
uuid: "2e204de3ce804cdaa3f588fa4a03e4aa"
format_stamp: "Formatted at 2026-08-09 06:25:58 on dist-test-slave-mjjr"
server_key: "9af6ee429c7cd560066ab63f162d95a4"
server_key_iv: "4f6990187b699685d9df1aae11e90380"
server_key_version: "encryptionkey@0"
I20260809 06:25:58.307415 19378 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:25:58.314446 19378 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 06:25:58.314702 19378 log.cc:1196] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 06:25:58.315294 19378 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: Deleting consensus metadata
I20260809 06:25:58.315604 19378 master_runner.cc:315] Copying system tablet from 127.8.131.62:45687
I20260809 06:25:58.318382 19378 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: tablet copy: Beginning tablet copy session from remote peer at address 127.8.131.62:45687
I20260809 06:25:58.335431 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.319281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60864 (local address 127.8.131.62:45687)
0809 06:25:58.319506 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.319512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.320256 (+ 744us) server_negotiation.cc:407] Connection header received
0809 06:25:58.320437 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.320444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.320533 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.320644 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.321700 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.322858 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.324844 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.325571 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.325703 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.328304 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.328310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.328315 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.328355 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.332323 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.333143 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.333150 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.333154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.333325 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.333780 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.333785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.333789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.334139 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.334281 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.334821 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.335063 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260809 06:25:58.335522 19683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.319229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:60864)
0809 06:25:58.320112 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.320132 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.320279 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.320696 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.320701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.321269 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.321514 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.321528 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.323040 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.323046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.324675 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.324693 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.325731 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.325737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.325924 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.326924 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.326948 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.328085 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.332456 (+ 4371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.332461 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.332466 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.333001 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.333445 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.333451 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.333454 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.333650 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.334260 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.334265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.334472 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.334628 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.334963 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":566,"threads_started":1}
I20260809 06:25:58.336386 18561 tablet_copy_service.cc:140] P 467cc2d070a8428d9cd4281d718a7875: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2e204de3ce804cdaa3f588fa4a03e4aa ({username='oryx', principal='oryx/127.8.131.60@KRBTEST.COM'} at 127.0.0.1:60864)
I20260809 06:25:58.336558 18561 tablet_copy_service.cc:161] P 467cc2d070a8428d9cd4281d718a7875: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2e204de3ce804cdaa3f588fa4a03e4aa at {username='oryx', principal='oryx/127.8.131.60@KRBTEST.COM'} at 127.0.0.1:60864: session id = 2e204de3ce804cdaa3f588fa4a03e4aa-00000000000000000000000000000000
I20260809 06:25:58.340323 18561 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 06:25:58.342235 19378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 06:25:58.349395 19378 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: tablet copy: Starting download of 0 data blocks...
I20260809 06:25:58.349725 19378 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: tablet copy: Starting download of 1 WAL segments...
I20260809 06:25:58.352322 19378 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 06:25:58.356859 18561 tablet_copy_service.cc:342] P 467cc2d070a8428d9cd4281d718a7875: Request end of tablet copy session 2e204de3ce804cdaa3f588fa4a03e4aa-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.131.60@KRBTEST.COM'} at 127.0.0.1:60864
I20260809 06:25:58.357013 18561 tablet_copy_service.cc:433] P 467cc2d070a8428d9cd4281d718a7875: ending tablet copy session 2e204de3ce804cdaa3f588fa4a03e4aa-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2e204de3ce804cdaa3f588fa4a03e4aa
I20260809 06:25:58.368794 19378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:58.373274 19684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:58.381116 19685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:58.388232 19689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:58.388537 19378 server_base.cc:1034] running on GCE node
I20260809 06:25:58.388958 19378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:58.389631 19378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:58.390858 19378 hybrid_clock.cc:648] HybridClock initialized: now 1786256758390804 us; error 36 us; skew 500 ppm
I20260809 06:25:58.391093 19378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256758, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:58.393348 19378 init.cc:377] Logged in from keytab as oryx/127.8.131.60@KRBTEST.COM (short username oryx)
I20260809 06:25:58.394758 19378 webserver.cc:460] Webserver started at http://127.8.131.60:39925/ using document root <none> and password file <none>
I20260809 06:25:58.395102 19378 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:58.395212 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:58.398448 19378 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 06:25:58.398959 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 18618
I20260809 06:25:58.404525 19702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:58.405085 19378 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260809 06:25:58.405222 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal
uuid: "2e204de3ce804cdaa3f588fa4a03e4aa"
format_stamp: "Formatted at 2026-08-09 06:25:58 on dist-test-slave-mjjr"
server_key: "9af6ee429c7cd560066ab63f162d95a4"
server_key_iv: "4f6990187b699685d9df1aae11e90380"
server_key_version: "encryptionkey@0"
I20260809 06:25:58.405411 19378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:58.414232 19378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:58.417752 19378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:58.427286 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:41977
--local_ip_for_outbound_sockets=127.8.131.1
--tserver_master_addrs=127.8.131.62:45687
--webserver_port=40165
--webserver_interface=127.8.131.1
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab
--principal=oryx/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.131.62:45687,127.8.131.61:33805
--tserver_master_addrs=127.8.131.62:45687,127.8.131.61:33805,127.8.131.60:39643 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:25:58.440421 19378 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.60:39643
I20260809 06:25:58.440493 19786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.60:39643 every 8 connection(s)
I20260809 06:25:58.441761 19378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
I20260809 06:25:58.447274 19787 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:25:58.451064 19787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: Bootstrap starting.
I20260809 06:25:58.470715 19787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:25:58.471452 19787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: Bootstrap complete.
I20260809 06:25:58.472573 19787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } }
I20260809 06:25:58.472872 19787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2e204de3ce804cdaa3f588fa4a03e4aa, State: Initialized, Role: LEARNER
I20260809 06:25:58.473143 19787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } }
I20260809 06:25:58.474080 19790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.474414 19787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:25:58.474475 19790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: This master's current role is: LEARNER
I20260809 06:25:58.483006 19806 catalog_manager.cc:1259] Loaded cluster ID: 08e563526049441699341a7c5f9b0abd
I20260809 06:25:58.483135 19806 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: loading cluster ID for follower catalog manager: success
I20260809 06:25:58.486770 19806 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: acquiring CA information for follower catalog manager: success
I20260809 06:25:58.489280 19806 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 06:25:58.796267 19781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:58.796615 19781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:58.796743 19781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:58.804332 19781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:58.804442 19781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:58.804569 19781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:25:58.814690 19781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab
--principal=oryx/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:41977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=40165
--tserver_master_addrs=127.8.131.62:45687,127.8.131.61:33805,127.8.131.60:39643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:58.816162 19781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:58.817485 19781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:58.832149 19811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:58.831955 19812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:58.831969 19815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:58.832780 19781 server_base.cc:1034] running on GCE node
I20260809 06:25:58.833459 19781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:58.834507 19781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:58.835767 19781 hybrid_clock.cc:648] HybridClock initialized: now 1786256758835721 us; error 33 us; skew 500 ppm
I20260809 06:25:58.836098 19781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256758, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:58.839509 19781 init.cc:377] Logged in from keytab as oryx/127.8.131.1@KRBTEST.COM (short username oryx)
I20260809 06:25:58.841187 19781 webserver.cc:460] Webserver started at http://127.8.131.1:40165/ using document root <none> and password file <none>
I20260809 06:25:58.841859 19781 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:58.841986 19781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:58.848316 19781 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 06:25:58.849851 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.843826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.8.131.60:39643)
0809 06:25:58.844378 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.844382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.844399 (+ 17us) server_negotiation.cc:407] Connection header received
0809 06:25:58.844460 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.844464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.844539 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.844635 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:25:58.845095 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.846098 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.848056 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.848910 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.849021 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.849168 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.849490 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.849587 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":284,"threads_started":1}
I20260809 06:25:58.849917 19822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.843505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:39643 (local address 127.0.0.1:57740)
0809 06:25:58.844026 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.844046 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.844182 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.844702 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.844708 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.844738 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:25:58.844936 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.844954 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.846253 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.846258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.847893 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.847909 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.849103 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.849110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.849310 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.849345 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.849462 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.849603 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":256,"threads_started":1}
I20260809 06:25:58.850924 19741 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 06:25:58.851773 19825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:58.851734 19790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 467cc2d070a8428d9cd4281d718a7875. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:58.852041 19790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: This master's current role is: LEARNER
I20260809 06:25:58.853248 19781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:25:58.853385 19781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "bb66859753634ddd90ab08678d482835"
format_stamp: "Formatted at 2026-08-09 06:25:55 on dist-test-slave-mjjr"
server_key: "ef76d7509839581939096a081aebecf5"
server_key_iv: "2731871e5fa31e8cd6806c2bdaf17479"
server_key_version: "encryptionkey@0"
I20260809 06:25:58.854329 19781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:58.871232 19781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:58.875378 19781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:58.875741 19781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:58.876894 19781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:58.878289 19781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:58.878415 19781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:58.878559 19781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:58.878674 19781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:58.912662 19781 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:41977
I20260809 06:25:58.912755 19996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:41977 every 8 connection(s)
I20260809 06:25:58.914163 19781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:25:58.923130 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 19781
I20260809 06:25:58.923540 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 18812
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256758, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256758, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.61@KRBTEST.COM
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256758, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.60@KRBTEST.COM
I20260809 06:25:58.946372 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.917432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:55745 (local address 127.8.131.61:33805)
0809 06:25:58.920086 (+ 2654us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.920092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.921373 (+ 1281us) server_negotiation.cc:407] Connection header received
0809 06:25:58.922413 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.922419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.922516 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.922697 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.925731 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.927250 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.930788 (+ 3538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.931539 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.931633 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.938285 (+ 6652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.938312 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.938316 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.938372 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.942230 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.943283 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.943289 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.943293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.943456 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.944024 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.944028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.944031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.944406 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.944547 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.945665 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.945905 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2522,"thread_start_us":2419,"threads_started":1}
I20260809 06:25:58.947582 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.917867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:52967 (local address 127.8.131.60:39643)
0809 06:25:58.918050 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.918056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.919190 (+ 1134us) server_negotiation.cc:407] Connection header received
0809 06:25:58.922306 (+ 3116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.922314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.922402 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.922481 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.931267 (+ 8786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.932430 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.935229 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.935865 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.936005 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.940583 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.940591 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.940598 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.940644 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.944305 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.945037 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.945044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.945047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.945198 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.945655 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.945659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.945662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.946017 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.946136 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.947024 (+ 888us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.947259 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:25:58.947619 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.917904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.1:49745)
0809 06:25:58.920644 (+ 2740us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.920662 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.922015 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.922997 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.923010 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.924122 (+ 1112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.925484 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.925504 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.927542 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.927550 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.930729 (+ 3179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.930750 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.932518 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.932527 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.932821 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.933635 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.933664 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.937601 (+ 3937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.943074 (+ 5473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.943080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.943084 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.943598 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.944122 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.944127 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.944130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.944289 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.945079 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.945083 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.945241 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.945508 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.946071 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2646,"mutex_wait_us":398,"spinlock_wait_cycles":30848,"thread_start_us":3018,"threads_started":1}
I20260809 06:25:58.947667 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.917649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.8.131.1:55745)
0809 06:25:58.921181 (+ 3532us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.921199 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.922020 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.922997 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.923008 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.924191 (+ 1183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.925469 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.925502 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.927506 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.927515 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.930613 (+ 3098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.930623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.931997 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.932004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.932270 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.933200 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.933231 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.938035 (+ 4804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.942423 (+ 4388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.942432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.942450 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.943152 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.943606 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.943610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.943613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.943882 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.944655 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.944663 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.945131 (+ 468us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.946700 (+ 1569us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.947028 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3414,"mutex_wait_us":710,"thread_start_us":2241,"threads_started":1}
I20260809 06:25:58.948729 20004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.917122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:49745 (local address 127.8.131.62:45687)
0809 06:25:58.917648 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.917655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.920761 (+ 3106us) server_negotiation.cc:407] Connection header received
0809 06:25:58.922306 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.922314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.922428 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.922591 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.925744 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.927302 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.930955 (+ 3653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.931902 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.932022 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.937905 (+ 5883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.937914 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.937920 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.937964 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.942401 (+ 4437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.943795 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.943801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.943804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.943976 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.944434 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.944441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.944444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.944864 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.945021 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.948063 (+ 3042us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.948308 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":582,"threads_started":1}
Aug 09 06:25:58 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.2@KRBTEST.COM
I20260809 06:25:58.949785 20000 heartbeater.cc:347] Connected to a master server at 127.8.131.61:33805
I20260809 06:25:58.950115 20000 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:58.950282 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.917650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:39643 (local address 127.8.131.1:52967)
0809 06:25:58.918997 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:58.919061 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:58.922018 (+ 2957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:58.922997 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:58.923008 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:58.930211 (+ 7203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:58.930525 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.930537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.932747 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.932753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.934998 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:58.935019 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.936504 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.936513 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:58.936727 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.937431 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:58.937453 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:58.940322 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:58.944462 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.944467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.944470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.944901 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.945370 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:58.945374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:58.945377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:58.945538 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:58.946225 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:58.946229 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:58.946423 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 06:25:58.949379 (+ 2956us) client_negotiation.cc:240] Negotiation successful
0809 06:25:58.949701 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":1,"thread_start_us":548,"threads_started":1}
I20260809 06:25:58.951329 20000 heartbeater.cc:511] Master 127.8.131.61:33805 requested a full tablet report, sending...
I20260809 06:25:58.951946 19999 heartbeater.cc:347] Connected to a master server at 127.8.131.60:39643
I20260809 06:25:58.952098 19999 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:58.952308 19998 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45687
I20260809 06:25:58.952456 19998 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:58.952723 19999 heartbeater.cc:511] Master 127.8.131.60:39643 requested a full tablet report, sending...
I20260809 06:25:58.953212 19998 heartbeater.cc:511] Master 127.8.131.62:45687 requested a full tablet report, sending...
I20260809 06:25:58.954535 18532 ts_manager.cc:194] Re-registered known tserver with Master: bb66859753634ddd90ab08678d482835 (127.8.131.1:41977)
I20260809 06:25:58.954876 19725 ts_manager.cc:194] Registered new tserver with Master: bb66859753634ddd90ab08678d482835 (127.8.131.1:41977)
I20260809 06:25:58.954984 19437 ts_manager.cc:194] Registered new tserver with Master: bb66859753634ddd90ab08678d482835 (127.8.131.1:41977)
I20260809 06:25:58.955708 18532 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:49745
I20260809 06:25:58.958600 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:58.924614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.8.131.2:36541)
0809 06:25:58.936218 (+ 11604us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:58.936225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:58.936250 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:25:58.936305 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:58.936311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:58.936620 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:58.936899 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:58.939383 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.941097 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.943092 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:58.944060 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:58.944166 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:58.950250 (+ 6084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:58.950266 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:58.950283 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:58.950352 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:58.954933 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.955995 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.956007 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.956013 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.956228 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:58.956755 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:58.956760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:58.956763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:58.957200 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:58.957529 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:58.957874 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 06:25:58.958157 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11430,"thread_start_us":11252,"threads_started":1}
I20260809 06:25:58.959852 19001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:59.045060 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 18812
I20260809 06:25:59.070020 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:36541
--local_ip_for_outbound_sockets=127.8.131.2
--tserver_master_addrs=127.8.131.62:45687
--webserver_port=37395
--webserver_interface=127.8.131.2
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab
--principal=oryx/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.131.62:45687,127.8.131.61:33805
--tserver_master_addrs=127.8.131.62:45687,127.8.131.61:33805,127.8.131.60:39643 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:25:59.267715 19658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: NON_VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: true } } }
I20260809 06:25:59.268179 19658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:59.270056 20019 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: attempting to promote NON_VOTER 2e204de3ce804cdaa3f588fa4a03e4aa to VOTER
I20260809 06:25:59.270658 20019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } }
I20260809 06:25:59.272552 19741 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 LEARNER]: Refusing update from remote peer 467cc2d070a8428d9cd4281d718a7875: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 06:25:59.272661 19448 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Refusing update from remote peer 467cc2d070a8428d9cd4281d718a7875: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 06:25:59.273288 19658 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 06:25:59.273669 20021 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 06:25:59.276393 20021 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2e204de3ce804cdaa3f588fa4a03e4aa (127.8.131.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:25:59.277070 19448 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2e204de3ce804cdaa3f588fa4a03e4aa (127.8.131.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:25:59.280746 20019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:25:59.281085 20019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:59.281443 19658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:25:59.281806 19658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:59.287349 20030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:25:59.287878 20030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:25:59.288992 19658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:25:59.289234 19658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875 [sys.catalog]: This master's current role is: LEADER
I20260809 06:25:59.290088 19741 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2e204de3ce804cdaa3f588fa4a03e4aa (127.8.131.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:25:59.293154 19790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "467cc2d070a8428d9cd4281d718a7875" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:25:59.293447 19790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: This master's current role is: FOLLOWER
W20260809 06:25:59.416544 20017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:25:59.416802 20017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:25:59.416903 20017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:25:59.424269 20017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:25:59.424376 20017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:25:59.424501 20017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:25:59.434664 20017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab
--principal=oryx/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:36541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=37395
--tserver_master_addrs=127.8.131.62:45687,127.8.131.61:33805,127.8.131.60:39643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:25:59.436019 20017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:25:59.437280 20017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:25:59.450920 20037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:59.450840 20039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:25:59.451769 20041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:25:59.452555 20017 server_base.cc:1034] running on GCE node
I20260809 06:25:59.453140 20017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:25:59.454046 20017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:25:59.455281 20017 hybrid_clock.cc:648] HybridClock initialized: now 1786256759455228 us; error 63 us; skew 500 ppm
I20260809 06:25:59.455588 20017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:25:59 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256759, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:25:59.458983 20017 init.cc:377] Logged in from keytab as oryx/127.8.131.2@KRBTEST.COM (short username oryx)
I20260809 06:25:59.460354 20017 webserver.cc:460] Webserver started at http://127.8.131.2:37395/ using document root <none> and password file <none>
I20260809 06:25:59.461076 20017 fs_manager.cc:359] Metadata directory not provided
I20260809 06:25:59.461225 20017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:25:59.466282 20017 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 06:25:59.468873 20054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:59.470207 20017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:25:59.470341 20017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "b583934b073a4f10a5acf20d959e17b7"
format_stamp: "Formatted at 2026-08-09 06:25:56 on dist-test-slave-mjjr"
server_key: "86948a03456cf222f89816d5e557e884"
server_key_iv: "aee6a2f0616d8d2a2552badce8b26d6f"
server_key_version: "encryptionkey@0"
I20260809 06:25:59.471238 20017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:25:59.493062 20017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:25:59.497308 20017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:25:59.497537 20017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:25:59.498442 20017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:25:59.499797 20017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:25:59.499910 20017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:59.499989 20017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:25:59.500042 20017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:25:59.536558 20017 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:36541
I20260809 06:25:59.536646 20209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:36541 every 8 connection(s)
I20260809 06:25:59.538024 20017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:25:59.545477 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 20017
I20260809 06:25:59.545805 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 19016
Aug 09 06:25:59 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.3@KRBTEST.COM
Aug 09 06:25:59 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256759, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.60@KRBTEST.COM
Aug 09 06:25:59 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256759, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.61@KRBTEST.COM
Aug 09 06:25:59 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256759, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:25:59.586750 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:59.547096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.8.131.3:39623)
0809 06:25:59.563276 (+ 16180us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:59.563283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:59.564289 (+ 1006us) server_negotiation.cc:407] Connection header received
0809 06:25:59.564498 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:59.564504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:59.564682 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:59.564873 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:59.567011 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.568713 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.570596 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.571470 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.571603 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:59.578129 (+ 6526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:59.578148 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:59.578167 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:59.578218 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:59.582230 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:59.583106 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:59.583115 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:59.583118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:59.583294 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:59.583962 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:59.583969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:59.583972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:59.584405 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:59.584738 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:59.585964 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0809 06:25:59.586251 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16032,"thread_start_us":16152,"threads_started":1}
I20260809 06:25:59.587414 20218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:59.540958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:39613 (local address 127.8.131.60:39643)
0809 06:25:59.541493 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:59.541501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:59.568585 (+ 27084us) server_negotiation.cc:407] Connection header received
0809 06:25:59.568692 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:59.568698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:59.568780 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:59.568885 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:59.569992 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.571129 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.573044 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.573517 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.573658 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:59.578726 (+ 5068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:59.578734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:59.578740 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:59.578787 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:59.583275 (+ 4488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:59.584453 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:59.584459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:59.584462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:59.584631 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:59.585229 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:59.585234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:59.585237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:59.585592 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:59.585726 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:59.586815 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0809 06:25:59.587040 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":529,"threads_started":1}
I20260809 06:25:59.588512 19205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:25:59.588635 20225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:59.542422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:39643 (local address 127.8.131.2:39613)
0809 06:25:59.568464 (+ 26042us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:59.568481 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:59.568587 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:59.569117 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:59.569122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:59.569620 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:59.569826 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:59.569834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.571302 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.571308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:59.572911 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:59.572920 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.573761 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.573771 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:59.574093 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:59.574799 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:59.574840 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:59.578483 (+ 3643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:59.583463 (+ 4980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:59.583472 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:59.583487 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:59.584314 (+ 827us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:59.584789 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:59.584793 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:59.584796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:59.585092 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:59.585816 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:59.585823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:59.586349 (+ 526us) client_negotiation.cc:769] Sending connection context
0809 06:25:59.587689 (+ 1340us) client_negotiation.cc:240] Negotiation successful
0809 06:25:59.587983 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25926,"mutex_wait_us":72,"spinlock_wait_cycles":12160,"thread_start_us":25772,"threads_started":1}
I20260809 06:25:59.589339 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:59.541308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:52275 (local address 127.8.131.61:33805)
0809 06:25:59.565094 (+ 23786us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:59.565103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:59.565127 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:25:59.565200 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:59.565208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:59.565312 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:59.565453 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:59.568939 (+ 3486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.570106 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.572819 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.573518 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.573666 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:59.579898 (+ 6232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:59.579908 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:59.579913 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:59.579960 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:59.583603 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:59.584500 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:59.584508 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:59.584511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:59.584679 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:59.585228 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:59.585234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:59.585237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:59.585593 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:59.585740 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:59.588732 (+ 2992us) server_negotiation.cc:299] Negotiation successful
0809 06:25:59.588978 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23628,"thread_start_us":25439,"threads_started":1}
I20260809 06:25:59.589673 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:59.541556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.8.131.2:52275)
0809 06:25:59.556040 (+ 14484us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:59.556105 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:59.560341 (+ 4236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:59.565730 (+ 5389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:59.565743 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:59.566605 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:59.568005 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:59.568044 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.570302 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.570308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:59.572661 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:59.572681 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.574117 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.574123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:59.574339 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:59.575136 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:59.575166 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:59.579598 (+ 4432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:59.583773 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:59.583777 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:59.583780 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:59.584360 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:59.584796 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:59.584801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:59.584804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:59.585100 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:59.585816 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:59.585823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:59.586349 (+ 526us) client_negotiation.cc:769] Sending connection context
0809 06:25:59.586686 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 06:25:59.587180 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14114,"mutex_wait_us":225,"spinlock_wait_cycles":12160,"thread_start_us":20098,"threads_started":1}
I20260809 06:25:59.590859 20216 heartbeater.cc:347] Connected to a master server at 127.8.131.61:33805
I20260809 06:25:59.591136 20216 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:59.591768 20212 heartbeater.cc:347] Connected to a master server at 127.8.131.60:39643
I20260809 06:25:59.591929 20212 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:59.592065 20216 heartbeater.cc:511] Master 127.8.131.61:33805 requested a full tablet report, sending...
I20260809 06:25:59.592492 20212 heartbeater.cc:511] Master 127.8.131.60:39643 requested a full tablet report, sending...
I20260809 06:25:59.593900 19437 ts_manager.cc:194] Registered new tserver with Master: b583934b073a4f10a5acf20d959e17b7 (127.8.131.2:36541)
I20260809 06:25:59.594206 19725 ts_manager.cc:194] Registered new tserver with Master: b583934b073a4f10a5acf20d959e17b7 (127.8.131.2:36541)
I20260809 06:25:59.595024 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:59.541874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:42747 (local address 127.8.131.62:45687)
0809 06:25:59.566027 (+ 24153us) server_negotiation.cc:206] Beginning negotiation
0809 06:25:59.566036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:25:59.566061 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:25:59.566137 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:25:59.566143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:25:59.566241 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:25:59.566371 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:25:59.568291 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.569374 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.574967 (+ 5593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.575811 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.575969 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:59.581375 (+ 5406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:25:59.581382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:25:59.581387 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:25:59.581428 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:25:59.585138 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:59.585986 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:59.585991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:59.585994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:59.586151 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:25:59.586604 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:25:59.586608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:25:59.586611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:25:59.586991 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:25:59.587117 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 06:25:59.594290 (+ 7173us) server_negotiation.cc:299] Negotiation successful
0809 06:25:59.594576 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23993,"thread_start_us":23888,"threads_started":1}
I20260809 06:25:59.595846 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:25:59.541556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.2:42747)
0809 06:25:59.561646 (+ 20090us) negotiation.cc:107] Waiting for socket to connect
0809 06:25:59.561660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 06:25:59.561748 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:25:59.566837 (+ 5089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:25:59.566841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:25:59.567613 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:25:59.568056 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:59.568069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.570317 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.570322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:59.574817 (+ 4495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:25:59.574835 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:25:59.576513 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:25:59.576529 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:25:59.576767 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:25:59.577487 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:25:59.577511 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:25:59.581146 (+ 3635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:25:59.585281 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:59.585286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:59.585289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:59.585833 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:59.586293 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:25:59.586298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:25:59.586301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:25:59.586446 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:25:59.587198 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:25:59.587203 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:25:59.587360 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 06:25:59.587491 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 06:25:59.587773 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19977,"mutex_wait_us":1321,"spinlock_wait_cycles":49024,"thread_start_us":16253,"threads_started":1}
I20260809 06:25:59.596849 20210 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45687
I20260809 06:25:59.596977 20210 heartbeater.cc:464] Registering TS with master...
I20260809 06:25:59.597386 20210 heartbeater.cc:511] Master 127.8.131.62:45687 requested a full tablet report, sending...
I20260809 06:25:59.598491 18532 ts_manager.cc:194] Re-registered known tserver with Master: b583934b073a4f10a5acf20d959e17b7 (127.8.131.2:36541)
I20260809 06:25:59.599320 18532 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:42747
I20260809 06:25:59.678135 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 19016
I20260809 06:25:59.702308 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:39623
--local_ip_for_outbound_sockets=127.8.131.3
--tserver_master_addrs=127.8.131.62:45687
--webserver_port=33633
--webserver_interface=127.8.131.3
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab
--principal=oryx/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.131.62:45687,127.8.131.61:33805
--tserver_master_addrs=127.8.131.62:45687,127.8.131.61:33805,127.8.131.60:39643 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:25:59.958686 19998 heartbeater.cc:503] Master 127.8.131.62:45687 was elected leader, sending a full tablet report...
W20260809 06:26:00.045747 20229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:00.046030 20229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:00.046126 20229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:00.053315 20229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:00.053429 20229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:00.053539 20229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:26:00.065140 20229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab
--principal=oryx/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:39623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=33633
--tserver_master_addrs=127.8.131.62:45687,127.8.131.61:33805,127.8.131.60:39643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:00.066532 20229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:00.067905 20229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:00.081911 20241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:00.082186 20243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:00.081962 20239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:00.082666 20229 server_base.cc:1034] running on GCE node
I20260809 06:26:00.083297 20229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:00.084252 20229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:00.085470 20229 hybrid_clock.cc:648] HybridClock initialized: now 1786256760085415 us; error 48 us; skew 500 ppm
I20260809 06:26:00.085769 20229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:00 dist-test-slave-mjjr krb5kdc[18470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256760, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:00.089130 20229 init.cc:377] Logged in from keytab as oryx/127.8.131.3@KRBTEST.COM (short username oryx)
I20260809 06:26:00.097110 20229 webserver.cc:460] Webserver started at http://127.8.131.3:33633/ using document root <none> and password file <none>
I20260809 06:26:00.097859 20229 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:00.098021 20229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:00.103441 20229 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:26:00.106326 20254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:00.107798 20229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:26:00.107973 20229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "470fe37812bf43fca27d30d336572ade"
format_stamp: "Formatted at 2026-08-09 06:25:56 on dist-test-slave-mjjr"
server_key: "9b76280d07ec66310e873bf094c82267"
server_key_iv: "750a09ddcfd4c87658f633ef569f6a47"
server_key_version: "encryptionkey@0"
I20260809 06:26:00.108960 20229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:00.122967 20229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:00.127311 20229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:00.127600 20229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:00.128722 20229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:00.130014 20229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:00.130101 20229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:00.130180 20229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:00.130235 20229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:00.164292 20229 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:39623
I20260809 06:26:00.164371 20414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:39623 every 8 connection(s)
I20260809 06:26:00.165822 20229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:26:00.166999 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 20229
Aug 09 06:26:00 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256760, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.61@KRBTEST.COM
Aug 09 06:26:00 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256760, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
Aug 09 06:26:00 dist-test-slave-mjjr krb5kdc[18470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256760, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.60@KRBTEST.COM
I20260809 06:26:00.221704 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.169844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.8.131.3:39447)
0809 06:26:00.186818 (+ 16974us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:00.186870 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:00.190133 (+ 3263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:00.197631 (+ 7498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:00.197635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:00.199573 (+ 1938us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:00.199864 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.199875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.200913 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.200920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.203111 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.203125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.204711 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.204721 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.205076 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.206209 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:00.206242 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:00.209548 (+ 3306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:00.213997 (+ 4449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:00.214006 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:00.214028 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:00.214772 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:00.215342 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:00.215347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:00.215352 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:00.215632 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:00.216440 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:00.216461 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:00.219157 (+ 2696us) client_negotiation.cc:769] Sending connection context
0809 06:26:00.219506 (+ 349us) client_negotiation.cc:240] Negotiation successful
0809 06:26:00.220094 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16629,"mutex_wait_us":66,"thread_start_us":27041,"threads_started":1}
I20260809 06:26:00.222913 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.170594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:39447 (local address 127.8.131.61:33805)
0809 06:26:00.193856 (+ 23262us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.193863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.193881 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:26:00.193939 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.193955 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.194036 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.194146 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.200022 (+ 5876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.200756 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.203659 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.204344 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.204450 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.209779 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.209784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.209787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.209833 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.213804 (+ 3971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.214928 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.214934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.214938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.215199 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.215799 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.215804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.215808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.216184 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.216351 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.222200 (+ 5849us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.222464 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23133,"thread_start_us":23025,"threads_started":1}
I20260809 06:26:00.225365 20419 heartbeater.cc:347] Connected to a master server at 127.8.131.61:33805
I20260809 06:26:00.225711 20419 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:00.226686 20419 heartbeater.cc:511] Master 127.8.131.61:33805 requested a full tablet report, sending...
I20260809 06:26:00.228641 19437 ts_manager.cc:194] Registered new tserver with Master: 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623)
I20260809 06:26:00.229552 20437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.194152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60868 (local address 127.8.131.62:45687)
0809 06:26:00.197785 (+ 3633us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.197794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.198555 (+ 761us) server_negotiation.cc:407] Connection header received
0809 06:26:00.198619 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.198626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.198713 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.198814 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.201630 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.203082 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.205443 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.206706 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.206847 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.220896 (+ 14049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.220904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.220912 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.220952 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.224807 (+ 3855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.225816 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.225821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.225825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.225987 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.226522 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.226527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.226533 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.226988 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.227247 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.228849 (+ 1602us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.229101 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3468,"thread_start_us":3326,"threads_started":1}
I20260809 06:26:00.234787 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.170970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.3:53265)
0809 06:26:00.193227 (+ 22257us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:00.193244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:00.193394 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:00.194653 (+ 1259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:00.194670 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:00.195957 (+ 1287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:00.197905 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.197930 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.200959 (+ 3029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.200966 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.203475 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.203485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.206974 (+ 3489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.206982 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.207155 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.208692 (+ 1537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:00.208719 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:00.216571 (+ 7852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:00.220803 (+ 4232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:00.220807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:00.220810 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:00.221380 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:00.227446 (+ 6066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:00.227455 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:00.227458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:00.227621 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:00.233480 (+ 5859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:00.233487 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:00.233688 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 06:26:00.233866 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 06:26:00.234278 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22155,"mutex_wait_us":975,"thread_start_us":21992,"threads_started":1}
I20260809 06:26:00.235283 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.169108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:53265 (local address 127.8.131.62:45687)
0809 06:26:00.191544 (+ 22436us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.191550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.193339 (+ 1789us) server_negotiation.cc:407] Connection header received
0809 06:26:00.193533 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.193540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.193650 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.193755 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.198085 (+ 4330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.199314 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.203607 (+ 4293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.204260 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.204392 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.216763 (+ 12371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.216770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.216775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.216815 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.220642 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.221525 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.221536 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.221540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.221972 (+ 432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.231778 (+ 9806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.231783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.231786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.232195 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.232362 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.234347 (+ 1985us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.234638 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22328,"thread_start_us":22249,"threads_started":1}
I20260809 06:26:00.236864 20415 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45687
I20260809 06:26:00.237025 20415 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:00.237548 20415 heartbeater.cc:511] Master 127.8.131.62:45687 requested a full tablet report, sending...
I20260809 06:26:00.241169 18531 ts_manager.cc:194] Re-registered known tserver with Master: 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623)
I20260809 06:26:00.242480 18531 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:53265
I20260809 06:26:00.242846 20428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.176953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:39643 (local address 127.8.131.3:45991)
0809 06:26:00.197387 (+ 20434us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:00.197411 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:00.197541 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:00.198418 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:00.198423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:00.198959 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:00.199219 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.199229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.204691 (+ 5462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.204696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.206794 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.206809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.211320 (+ 4511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.211327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.211515 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.213130 (+ 1615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:00.213151 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:00.215842 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:00.233459 (+ 17617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:00.233466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:00.233472 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:00.234072 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:00.237264 (+ 3192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:00.237272 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:00.237279 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:00.237517 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:00.241831 (+ 4314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:00.241836 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:00.241992 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 06:26:00.242137 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 06:26:00.242444 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20270,"thread_start_us":9397,"threads_started":1}
I20260809 06:26:00.245879 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.177550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:45991 (local address 127.8.131.60:39643)
0809 06:26:00.191366 (+ 13816us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.191373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.198098 (+ 6725us) server_negotiation.cc:407] Connection header received
0809 06:26:00.198169 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.198176 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.198257 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.199378 (+ 1121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.199570 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.202274 (+ 2704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.207527 (+ 5253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.210880 (+ 3353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.211045 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.216763 (+ 5718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.216770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.216775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.216815 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.233274 (+ 16459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.236893 (+ 3619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.236899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.236902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.237081 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.237732 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.237737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.237740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.238114 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.238254 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.245224 (+ 6970us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.245466 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13672,"thread_start_us":13567,"threads_started":1}
I20260809 06:26:00.246670 20418 heartbeater.cc:347] Connected to a master server at 127.8.131.60:39643
I20260809 06:26:00.246812 20418 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:00.247274 20418 heartbeater.cc:511] Master 127.8.131.60:39643 requested a full tablet report, sending...
I20260809 06:26:00.247896 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.199257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.8.131.60:39643)
0809 06:26:00.220214 (+ 20957us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.220223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.220254 (+ 31us) server_negotiation.cc:407] Connection header received
0809 06:26:00.220316 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.220325 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.220428 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.220547 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.222761 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.224246 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.228454 (+ 4208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.229278 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.229406 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.235162 (+ 5756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.235169 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.235173 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.235218 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.241383 (+ 6165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.242232 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.242238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.242242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.242400 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.242910 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.245265 (+ 2355us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.245269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.245616 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.247131 (+ 1515us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.247272 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.247499 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20759,"thread_start_us":20627,"threads_started":1}
I20260809 06:26:00.248508 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.208352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44132 (local address 127.8.131.61:33805)
0809 06:26:00.217297 (+ 8945us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.217302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.222533 (+ 5231us) server_negotiation.cc:407] Connection header received
0809 06:26:00.222592 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.222597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.222689 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.222790 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.224956 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.226161 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.228576 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.229273 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.229395 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.238017 (+ 8622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.238028 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.238036 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.238083 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.243842 (+ 5759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.244773 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.244779 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.244782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.244985 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.245842 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.245850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.245854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.246207 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.246514 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.247685 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.247925 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8812,"thread_start_us":11332,"threads_started":1}
I20260809 06:26:00.248741 19725 ts_manager.cc:194] Registered new tserver with Master: 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623)
I20260809 06:26:00.239156 18532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:26:00.251055 18532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:26:00.293084 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.271419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36824 (local address 127.8.131.2:36541)
0809 06:26:00.279569 (+ 8150us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.279582 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.283625 (+ 4043us) server_negotiation.cc:407] Connection header received
0809 06:26:00.283863 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.283874 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.284242 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.284746 (+ 504us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:00.285964 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.287929 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.290529 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.292193 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.292290 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.292453 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.292739 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.292870 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7920,"thread_start_us":7752,"threads_started":1}
I20260809 06:26:00.293290 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.270799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:36541 (local address 127.0.0.1:36824)
0809 06:26:00.283342 (+ 12543us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:00.283367 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:00.283674 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:00.285153 (+ 1479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:00.285161 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:00.285200 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:00.285705 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.285721 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.288267 (+ 2546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.288273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.290347 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.290361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.292366 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.292373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.292561 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.292597 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 06:26:00.292696 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 06:26:00.292823 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12376,"mutex_wait_us":1,"thread_start_us":1481,"threads_started":1}
I20260809 06:26:00.296263 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.270624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.8.131.3:39623)
0809 06:26:00.288243 (+ 17619us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.288252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.288284 (+ 32us) server_negotiation.cc:407] Connection header received
0809 06:26:00.288380 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.288388 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.288609 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.288949 (+ 340us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:00.289680 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.291234 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.293675 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.295197 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.295331 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.295532 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.295830 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.296005 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17446,"thread_start_us":17275,"threads_started":1}
I20260809 06:26:00.296262 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.270316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:39623 (local address 127.0.0.1:52990)
0809 06:26:00.272574 (+ 2258us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:00.272599 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:00.280213 (+ 7614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:00.288950 (+ 8737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:00.288957 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:00.288991 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:00.289435 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.289461 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.291363 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.291368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.293517 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.293532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.295433 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.295439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.295608 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.295666 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 06:26:00.295758 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 06:26:00.295897 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2100,"thread_start_us":12683,"threads_started":1}
I20260809 06:26:00.297048 20123 tablet_service.cc:1467] Processing CreateTablet for tablet eded359207304e1690ed1eaa0e334938 (DEFAULT_TABLE table=test-table [id=c2e6094c45a940ccabbc2ad23b378fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:00.298718 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eded359207304e1690ed1eaa0e334938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:00.299960 20317 tablet_service.cc:1467] Processing CreateTablet for tablet eded359207304e1690ed1eaa0e334938 (DEFAULT_TABLE table=test-table [id=c2e6094c45a940ccabbc2ad23b378fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:00.301349 20317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eded359207304e1690ed1eaa0e334938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:00.306172 20458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.272793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54094)
0809 06:26:00.287932 (+ 15139us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:00.287950 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:00.288267 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:00.289263 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:00.289266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:00.289287 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:00.289772 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.289787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.293805 (+ 4018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.293811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.295889 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.295904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.305488 (+ 9584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.305493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.305626 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.305650 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 06:26:00.305762 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 06:26:00.305934 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15024,"thread_start_us":15572,"threads_started":1}
I20260809 06:26:00.307062 20459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.273251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54094 (local address 127.8.131.1:41977)
0809 06:26:00.284928 (+ 11677us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.284940 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.288256 (+ 3316us) server_negotiation.cc:407] Connection header received
0809 06:26:00.288412 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.288419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.288607 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.288942 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:00.289973 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.293670 (+ 3697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.303750 (+ 10080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.305290 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.305440 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.305656 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.306531 (+ 875us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.306659 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11421,"thread_start_us":11679,"threads_started":1}
I20260809 06:26:00.310083 19897 tablet_service.cc:1467] Processing CreateTablet for tablet eded359207304e1690ed1eaa0e334938 (DEFAULT_TABLE table=test-table [id=c2e6094c45a940ccabbc2ad23b378fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:00.311213 19897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eded359207304e1690ed1eaa0e334938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:00.317068 20460 tablet_bootstrap.cc:492] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: Bootstrap starting.
I20260809 06:26:00.317912 20461 tablet_bootstrap.cc:492] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: Bootstrap starting.
I20260809 06:26:00.321691 20461 tablet_bootstrap.cc:654] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:00.322250 20460 tablet_bootstrap.cc:654] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:00.323048 20461 log.cc:824] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:00.323451 20460 log.cc:824] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:00.326350 20461 tablet_bootstrap.cc:492] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: No bootstrap required, opened a new log
I20260809 06:26:00.326646 20461 ts_tablet_manager.cc:1397] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 06:26:00.326687 20460 tablet_bootstrap.cc:492] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: No bootstrap required, opened a new log
I20260809 06:26:00.326932 20460 ts_tablet_manager.cc:1397] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 06:26:00.327836 20465 tablet_bootstrap.cc:492] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: Bootstrap starting.
I20260809 06:26:00.331560 20465 tablet_bootstrap.cc:654] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:00.332940 20465 log.cc:824] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:00.336299 20465 tablet_bootstrap.cc:492] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: No bootstrap required, opened a new log
I20260809 06:26:00.336167 20460 raft_consensus.cc:348] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } }
I20260809 06:26:00.336591 20465 ts_tablet_manager.cc:1397] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 06:26:00.336598 20460 raft_consensus.cc:374] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:00.336696 20460 raft_consensus.cc:729] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b583934b073a4f10a5acf20d959e17b7, State: Initialized, Role: FOLLOWER
I20260809 06:26:00.337316 20460 consensus_queue.cc:260] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } }
I20260809 06:26:00.337426 20461 raft_consensus.cc:348] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } }
I20260809 06:26:00.337893 20461 raft_consensus.cc:374] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:00.338016 20461 raft_consensus.cc:729] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470fe37812bf43fca27d30d336572ade, State: Initialized, Role: FOLLOWER
I20260809 06:26:00.338729 20210 heartbeater.cc:503] Master 127.8.131.62:45687 was elected leader, sending a full tablet report...
I20260809 06:26:00.338871 20461 consensus_queue.cc:260] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } }
I20260809 06:26:00.339300 20460 ts_tablet_manager.cc:1428] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260809 06:26:00.344233 20470 raft_consensus.cc:482] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:26:00.344530 20470 raft_consensus.cc:504] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } }
I20260809 06:26:00.350764 20465 raft_consensus.cc:348] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } }
I20260809 06:26:00.351312 20465 raft_consensus.cc:374] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:00.351440 20465 raft_consensus.cc:729] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb66859753634ddd90ab08678d482835, State: Initialized, Role: FOLLOWER
I20260809 06:26:00.352638 20415 heartbeater.cc:503] Master 127.8.131.62:45687 was elected leader, sending a full tablet report...
I20260809 06:26:00.353036 20465 consensus_queue.cc:260] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } }
I20260809 06:26:00.353935 20470 leader_election.cc:288] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623), bb66859753634ddd90ab08678d482835 (127.8.131.1:41977)
I20260809 06:26:00.358186 20461 ts_tablet_manager.cc:1428] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: Time spent starting tablet: real 0.031s user 0.023s sys 0.004s
I20260809 06:26:00.368376 20465 ts_tablet_manager.cc:1428] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: Time spent starting tablet: real 0.032s user 0.015s sys 0.009s
I20260809 06:26:00.370517 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.348781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:54199 (local address 127.8.131.3:39623)
0809 06:26:00.349023 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.349032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.362547 (+ 13515us) server_negotiation.cc:407] Connection header received
0809 06:26:00.363890 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.363896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.363986 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.364056 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:00.364906 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.365998 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.368221 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.369498 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.369652 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.369760 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.370121 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.370267 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:26:00.371218 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.348181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:39623 (local address 127.8.131.2:54199)
0809 06:26:00.362281 (+ 14100us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:00.362304 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:00.363705 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:00.364265 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:00.364269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:00.364298 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:00.364732 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.364748 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.366186 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.366190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.368067 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.368079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.369711 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.369717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.369897 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.369932 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 06:26:00.370711 (+ 779us) client_negotiation.cc:240] Negotiation successful
0809 06:26:00.370851 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13960,"thread_start_us":14367,"threads_started":1}
I20260809 06:26:00.372196 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eded359207304e1690ed1eaa0e334938" candidate_uuid: "b583934b073a4f10a5acf20d959e17b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470fe37812bf43fca27d30d336572ade" is_pre_election: true
I20260809 06:26:00.372782 20345 raft_consensus.cc:2454] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b583934b073a4f10a5acf20d959e17b7 in term 0.
I20260809 06:26:00.373060 20459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.347914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:41675 (local address 127.8.131.1:41977)
0809 06:26:00.348113 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.348120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.363359 (+ 15239us) server_negotiation.cc:407] Connection header received
0809 06:26:00.363832 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.363838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.363937 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.364000 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:00.364909 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.366975 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.370779 (+ 3804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.371940 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.372056 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.372157 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.372739 (+ 582us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.372842 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1154,"server-negotiator.queue_time_us":58}
I20260809 06:26:00.373261 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.348305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.8.131.2:41675)
0809 06:26:00.363119 (+ 14814us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:00.363139 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:00.363633 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:00.364206 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:00.364210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:00.364236 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:00.364685 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.364700 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.367133 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.367141 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.370560 (+ 3419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:00.370590 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.372378 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.372394 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:00.372585 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.372617 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 06:26:00.372713 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 06:26:00.372847 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14698,"thread_start_us":15088,"threads_started":1}
I20260809 06:26:00.373541 20057 leader_election.cc:302] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 470fe37812bf43fca27d30d336572ade, b583934b073a4f10a5acf20d959e17b7; no voters:
I20260809 06:26:00.373997 20470 raft_consensus.cc:2781] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:26:00.374138 20470 raft_consensus.cc:482] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:26:00.374271 20470 raft_consensus.cc:3037] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:00.374372 19926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eded359207304e1690ed1eaa0e334938" candidate_uuid: "b583934b073a4f10a5acf20d959e17b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb66859753634ddd90ab08678d482835" is_pre_election: true
I20260809 06:26:00.374893 19926 raft_consensus.cc:2454] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b583934b073a4f10a5acf20d959e17b7 in term 0.
I20260809 06:26:00.377121 20470 raft_consensus.cc:504] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } }
I20260809 06:26:00.377916 20470 leader_election.cc:288] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [CANDIDATE]: Term 1 election: Requested vote from peers 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623), bb66859753634ddd90ab08678d482835 (127.8.131.1:41977)
I20260809 06:26:00.378306 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eded359207304e1690ed1eaa0e334938" candidate_uuid: "b583934b073a4f10a5acf20d959e17b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470fe37812bf43fca27d30d336572ade"
I20260809 06:26:00.378361 19926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eded359207304e1690ed1eaa0e334938" candidate_uuid: "b583934b073a4f10a5acf20d959e17b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb66859753634ddd90ab08678d482835"
I20260809 06:26:00.378518 20345 raft_consensus.cc:3037] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:00.378572 19926 raft_consensus.cc:3037] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:00.380986 19926 raft_consensus.cc:2454] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b583934b073a4f10a5acf20d959e17b7 in term 1.
I20260809 06:26:00.381101 20345 raft_consensus.cc:2454] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b583934b073a4f10a5acf20d959e17b7 in term 1.
I20260809 06:26:00.381413 20058 leader_election.cc:302] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b583934b073a4f10a5acf20d959e17b7, bb66859753634ddd90ab08678d482835; no voters:
I20260809 06:26:00.381769 20470 raft_consensus.cc:2781] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:00.382203 20470 raft_consensus.cc:686] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 1 LEADER]: Becoming Leader. State: Replica: b583934b073a4f10a5acf20d959e17b7, State: Running, Role: LEADER
I20260809 06:26:00.382759 20470 consensus_queue.cc:237] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } }
I20260809 06:26:00.387252 18532 catalog_manager.cc:5495] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b583934b073a4f10a5acf20d959e17b7 (127.8.131.2). New cstate: current_term: 1 leader_uuid: "b583934b073a4f10a5acf20d959e17b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } health_report { overall_health: UNKNOWN } } }
I20260809 06:26:00.408015 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.402491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.8.131.2:36541)
0809 06:26:00.402737 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.402744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.402844 (+ 100us) server_negotiation.cc:407] Connection header received
0809 06:26:00.403085 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.403089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.403163 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.403314 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:00.403730 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.404441 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.405804 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.406310 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.406458 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.406944 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:00.407400 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:00.407561 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.407661 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.407740 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 06:26:00.415793 20345 raft_consensus.cc:1261] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [term 1 FOLLOWER]: Refusing update from remote peer b583934b073a4f10a5acf20d959e17b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:00.415849 19926 raft_consensus.cc:1261] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [term 1 FOLLOWER]: Refusing update from remote peer b583934b073a4f10a5acf20d959e17b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:00.416759 20470 consensus_queue.cc:1035] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:00.417029 20475 consensus_queue.cc:1035] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 06:26:00.421691 20003 tablet.cc:2369] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:26:00.439113 20480 mvcc.cc:205] Tried to move back new op lower bound from 7316507690653925376 to 7316507690531807232. Current Snapshot: MvccSnapshot[applied={T|T < 7316507690653925376}]
I20260809 06:26:00.457367 18532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:00.457664 18532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:00.464306 18532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 467cc2d070a8428d9cd4281d718a7875: Sending DeleteTablet for 3 replicas of tablet eded359207304e1690ed1eaa0e334938
I20260809 06:26:00.465749 19897 tablet_service.cc:1514] Processing DeleteTablet for tablet eded359207304e1690ed1eaa0e334938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:00 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:54094
I20260809 06:26:00.465759 20123 tablet_service.cc:1514] Processing DeleteTablet for tablet eded359207304e1690ed1eaa0e334938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:00 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:36824
I20260809 06:26:00.466517 20317 tablet_service.cc:1514] Processing DeleteTablet for tablet eded359207304e1690ed1eaa0e334938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:00 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:52990
I20260809 06:26:00.466878 20493 tablet_replica.cc:330] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: stopping tablet replica
I20260809 06:26:00.467515 20493 raft_consensus.cc:2229] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:26:00.468642 20492 tablet_replica.cc:330] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: stopping tablet replica
I20260809 06:26:00.469026 20492 raft_consensus.cc:2229] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:00.469527 20492 raft_consensus.cc:2258] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:00.470131 20493 raft_consensus.cc:2258] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:00.472635 20493 ts_tablet_manager.cc:1905] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:00.476619 20492 ts_tablet_manager.cc:1905] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:00.478089 20495 tablet_replica.cc:330] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: stopping tablet replica
I20260809 06:26:00.478508 20495 raft_consensus.cc:2229] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:00.479041 20495 raft_consensus.cc:2258] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:00.481045 20495 ts_tablet_manager.cc:1905] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:00.484584 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 18484
I20260809 06:26:00.484612 20492 ts_tablet_manager.cc:1918] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:00.484779 20492 log.cc:1196] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/eded359207304e1690ed1eaa0e334938
I20260809 06:26:00.485658 18574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:00.489269 20495 ts_tablet_manager.cc:1918] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:00.489454 20495 log.cc:1196] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/eded359207304e1690ed1eaa0e334938
I20260809 06:26:00.490051 20495 ts_tablet_manager.cc:1939] T eded359207304e1690ed1eaa0e334938 P 470fe37812bf43fca27d30d336572ade: Deleting consensus metadata
I20260809 06:26:00.498193 20493 ts_tablet_manager.cc:1918] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:00.498373 20493 log.cc:1196] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/eded359207304e1690ed1eaa0e334938
I20260809 06:26:00.499118 20493 ts_tablet_manager.cc:1939] T eded359207304e1690ed1eaa0e334938 P b583934b073a4f10a5acf20d959e17b7: Deleting consensus metadata
I20260809 06:26:00.500554 20492 ts_tablet_manager.cc:1939] T eded359207304e1690ed1eaa0e334938 P bb66859753634ddd90ab08678d482835: Deleting consensus metadata
I20260809 06:26:00.504595 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.470377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44142 (local address 127.8.131.61:33805)
0809 06:26:00.470578 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.470585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.482814 (+ 12229us) server_negotiation.cc:407] Connection header received
0809 06:26:00.484353 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.484360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.484455 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.484537 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.488213 (+ 3676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.489690 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.492215 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.492853 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.492971 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.496715 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.496722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.496728 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.496769 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.500566 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.501534 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.501542 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.501548 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.501729 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.502547 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.502555 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.502561 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.502977 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.503305 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.504016 (+ 711us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.504237 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:26:00.522789 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.483338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57756 (local address 127.8.131.60:39643)
0809 06:26:00.483513 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.483519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.505533 (+ 22014us) server_negotiation.cc:407] Connection header received
0809 06:26:00.505777 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.505782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.505883 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.505959 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.507896 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.509092 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.510960 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.511707 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.511806 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.515528 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.515534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.515538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.515582 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.519205 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.520164 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.520171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.520176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.520338 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.520864 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.520870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.520874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.521277 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.521563 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.522143 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.522427 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 06:26:00.647985 18512 catalog_manager.cc:4841] TS 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623): tablet eded359207304e1690ed1eaa0e334938 (table test-table [id=c2e6094c45a940ccabbc2ad23b378fc9]) successfully deleted
W20260809 06:26:00.648487 18512 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.131.3:39623, user_credentials={real_user=slave}} blocked reactor thread for 156519us
I20260809 06:26:00.648870 18514 catalog_manager.cc:4841] TS b583934b073a4f10a5acf20d959e17b7 (127.8.131.2:36541): tablet eded359207304e1690ed1eaa0e334938 (table test-table [id=c2e6094c45a940ccabbc2ad23b378fc9]) successfully deleted
W20260809 06:26:00.649044 18514 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.131.2:36541, user_credentials={real_user=slave}} blocked reactor thread for 148054us
I20260809 06:26:00.649330 18514 catalog_manager.cc:4841] TS bb66859753634ddd90ab08678d482835 (127.8.131.1:41977): tablet eded359207304e1690ed1eaa0e334938 (table test-table [id=c2e6094c45a940ccabbc2ad23b378fc9]) successfully deleted
I20260809 06:26:00.649402 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 18484
I20260809 06:26:00.724434 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.696128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.8.131.60:39643)
0809 06:26:00.696323 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.696331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.708145 (+ 11814us) server_negotiation.cc:407] Connection header received
0809 06:26:00.708336 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.708342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.708432 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.708540 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.710796 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.711903 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.713880 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.714308 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.714457 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.717396 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.717406 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.717409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.717443 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.721205 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.722121 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.722130 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.722134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.722294 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.722803 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.722809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.722815 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.723250 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.723528 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.723915 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.724103 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 06:26:00.726130 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:00.693081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44158 (local address 127.8.131.61:33805)
0809 06:26:00.693277 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:00.693283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:00.704457 (+ 11174us) server_negotiation.cc:407] Connection header received
0809 06:26:00.704700 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:00.704706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:00.704805 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:00.704918 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:00.710209 (+ 5291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.711508 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.713829 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:00.714509 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:00.714659 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:00.718357 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:00.718384 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:00.718388 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:00.718433 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:00.722014 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.722919 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.722925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.722929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.723091 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:00.723585 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:00.723591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:00.723595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:00.724037 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:00.724337 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:00.725579 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0809 06:26:00.725803 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 06:26:01.436744 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.435731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.3:45883)
0809 06:26:01.436319 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.436497 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":319,"threads_started":1}
W20260809 06:26:01.437182 20415 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:45687 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
I20260809 06:26:01.441989 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.441212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.2:49275)
0809 06:26:01.441729 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.441838 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":307,"threads_started":1}
W20260809 06:26:01.442416 20210 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:45687 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
I20260809 06:26:01.448726 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.447620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.1:33817)
0809 06:26:01.448377 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.448525 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":499,"threads_started":1}
W20260809 06:26:01.449261 19998 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:45687 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
I20260809 06:26:01.964742 20513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 467cc2d070a8428d9cd4281d718a7875)
I20260809 06:26:01.965209 20514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 467cc2d070a8428d9cd4281d718a7875)
I20260809 06:26:01.965126 20513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } }
I20260809 06:26:01.965483 20514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } }
I20260809 06:26:01.966699 20513 leader_election.cc:288] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 467cc2d070a8428d9cd4281d718a7875 (127.8.131.62:45687), 2e204de3ce804cdaa3f588fa4a03e4aa (127.8.131.60:39643)
I20260809 06:26:01.966851 20514 leader_election.cc:288] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 467cc2d070a8428d9cd4281d718a7875 (127.8.131.62:45687), 52b995272cdd470b9dd9af9cffbc4879 (127.8.131.61:33805)
I20260809 06:26:01.972028 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.966867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33084)
0809 06:26:01.971528 (+ 4661us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.971758 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4517,"threads_started":1}
I20260809 06:26:01.972468 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.967592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33100)
0809 06:26:01.971856 (+ 4264us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.972073 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4141,"thread_start_us":3964,"threads_started":1}
W20260809 06:26:01.972880 19704 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
W20260809 06:26:01.974160 19409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
I20260809 06:26:01.976276 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.975750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33118)
0809 06:26:01.976042 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.976127 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
W20260809 06:26:01.976440 19704 leader_election.cc:334] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 467cc2d070a8428d9cd4281d718a7875 (127.8.131.62:45687): Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
I20260809 06:26:01.976778 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.976010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33126)
0809 06:26:01.976253 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.976481 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260809 06:26:01.976999 19409 leader_election.cc:334] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 467cc2d070a8428d9cd4281d718a7875 (127.8.131.62:45687): Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
I20260809 06:26:01.980769 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.966970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:39643 (local address 127.0.0.1:57776)
0809 06:26:01.972222 (+ 5252us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.972240 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:01.972360 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:01.973305 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:01.973312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:01.973343 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:01.973693 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:01.973712 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:01.975996 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:01.976000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:01.978380 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:01.978391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:01.980019 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:01.980023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:01.980150 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:01.980176 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 06:26:01.980253 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 06:26:01.980349 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5167,"mutex_wait_us":27,"thread_start_us":4256,"threads_started":1}
I20260809 06:26:01.980880 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.971865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.8.131.60:39643)
0809 06:26:01.972940 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:01.972947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:01.972969 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:26:01.973043 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:01.973046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:01.973140 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:01.973958 (+ 818us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:01.974240 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:01.975804 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:01.978539 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:01.979856 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:01.980004 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:01.980100 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:01.980301 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 06:26:01.980510 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":889,"thread_start_us":749,"threads_started":1}
I20260809 06:26:01.981500 19741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "52b995272cdd470b9dd9af9cffbc4879" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" is_pre_election: true
I20260809 06:26:01.981848 19741 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b995272cdd470b9dd9af9cffbc4879 in term 1.
I20260809 06:26:01.982338 19410 leader_election.cc:302] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e204de3ce804cdaa3f588fa4a03e4aa, 52b995272cdd470b9dd9af9cffbc4879; no voters: 467cc2d070a8428d9cd4281d718a7875
I20260809 06:26:01.982774 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.967195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:33805 (local address 127.0.0.1:44166)
0809 06:26:01.967899 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.967921 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:01.968141 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:01.975419 (+ 7278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:01.975424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:01.975450 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:01.975804 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:01.975816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:01.978255 (+ 2439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:01.978259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:01.980443 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:01.980456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:01.981891 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:01.981900 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:01.982086 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:01.982119 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 06:26:01.982218 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 06:26:01.982358 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":83,"thread_start_us":619,"threads_started":1}
I20260809 06:26:01.982780 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.973450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.8.131.61:33805)
0809 06:26:01.974278 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:01.974287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:01.974310 (+ 23us) server_negotiation.cc:407] Connection header received
0809 06:26:01.974388 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:01.974393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:01.974487 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:01.976000 (+ 1513us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:01.976482 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:01.978107 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:01.980654 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:01.981647 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:01.981800 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:01.981920 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:01.982261 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 06:26:01.982393 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":451,"threads_started":1}
I20260809 06:26:01.982786 20513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 06:26:01.982949 20513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 467cc2d070a8428d9cd4281d718a7875)
I20260809 06:26:01.983048 20513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:26:01.983338 19448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "52b995272cdd470b9dd9af9cffbc4879" is_pre_election: true
I20260809 06:26:01.985371 20513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } }
I20260809 06:26:01.985812 19448 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2e204de3ce804cdaa3f588fa4a03e4aa in current term 2: Already voted for candidate 52b995272cdd470b9dd9af9cffbc4879 in this term.
I20260809 06:26:01.986176 20513 leader_election.cc:288] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [CANDIDATE]: Term 2 election: Requested vote from peers 467cc2d070a8428d9cd4281d718a7875 (127.8.131.62:45687), 2e204de3ce804cdaa3f588fa4a03e4aa (127.8.131.60:39643)
I20260809 06:26:01.986624 19741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "52b995272cdd470b9dd9af9cffbc4879" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa"
I20260809 06:26:01.986891 19741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:26:01.986963 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.986563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33138)
0809 06:26:01.986728 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.986820 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 06:26:01.987089 19707 leader_election.cc:302] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2e204de3ce804cdaa3f588fa4a03e4aa; no voters: 467cc2d070a8428d9cd4281d718a7875, 52b995272cdd470b9dd9af9cffbc4879
I20260809 06:26:01.988557 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:01.988218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33150)
0809 06:26:01.988372 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:01.988455 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 06:26:01.988750 19409 leader_election.cc:334] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 467cc2d070a8428d9cd4281d718a7875 (127.8.131.62:45687): Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
I20260809 06:26:01.989655 19741 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b995272cdd470b9dd9af9cffbc4879 in term 2.
I20260809 06:26:01.989902 20514 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 06:26:01.990113 19410 leader_election.cc:302] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e204de3ce804cdaa3f588fa4a03e4aa, 52b995272cdd470b9dd9af9cffbc4879; no voters: 467cc2d070a8428d9cd4281d718a7875
I20260809 06:26:01.990415 20513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 06:26:01.990907 20513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [term 2 LEADER]: Becoming Leader. State: Replica: 52b995272cdd470b9dd9af9cffbc4879, State: Running, Role: LEADER
I20260809 06:26:01.991291 20513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } }
I20260809 06:26:01.993206 20528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52b995272cdd470b9dd9af9cffbc4879. Latest consensus state: current_term: 2 leader_uuid: "52b995272cdd470b9dd9af9cffbc4879" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:26:01.993517 20528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:01.994199 20530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:26:01.998101 20530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:26:01.998931 20530 catalog_manager.cc:1259] Loaded cluster ID: 08e563526049441699341a7c5f9b0abd
I20260809 06:26:01.999046 20530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:26:02.000334 20530 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:26:02.001178 20530 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Loaded TSK: 0
I20260809 06:26:02.002108 20530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:26:02.031747 19437 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:26:02.038733 19437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:26:02.042918 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.042502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33164)
0809 06:26:02.042702 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.042781 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 06:26:02.043048 19741 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [term 2 FOLLOWER]: Refusing update from remote peer 52b995272cdd470b9dd9af9cffbc4879: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 06:26:02.043859 20513 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 06:26:02.044045 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.043585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33166)
0809 06:26:02.043806 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.043914 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 06:26:02.044225 19409 consensus_peers.cc:487] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 -> Peer 467cc2d070a8428d9cd4281d718a7875 (127.8.131.62:45687): Couldn't send request to peer 467cc2d070a8428d9cd4281d718a7875. Status: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 06:26:02.046607 20514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52b995272cdd470b9dd9af9cffbc4879. Latest consensus state: current_term: 2 leader_uuid: "52b995272cdd470b9dd9af9cffbc4879" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:26:02.046903 20514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:26:02.048575 20513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "52b995272cdd470b9dd9af9cffbc4879" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:26:02.048913 20513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879 [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:02.049144 20514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "52b995272cdd470b9dd9af9cffbc4879" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "467cc2d070a8428d9cd4281d718a7875" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45687 } } peers { permanent_uuid: "52b995272cdd470b9dd9af9cffbc4879" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 33805 } attrs { promote: false } } peers { permanent_uuid: "2e204de3ce804cdaa3f588fa4a03e4aa" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 39643 } attrs { promote: false } } }
I20260809 06:26:02.049448 20514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e204de3ce804cdaa3f588fa4a03e4aa [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:26:02.064464 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.056936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:39623 (local address 127.0.0.1:52994)
0809 06:26:02.057121 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.057141 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:02.057263 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:02.058048 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:02.058055 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:02.058081 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:02.058265 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:02.058278 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.060056 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.060063 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:02.062044 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:02.062059 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.063677 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.063685 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:02.063893 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.063929 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 06:26:02.064008 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 06:26:02.064149 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 06:26:02.065655 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.057413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54102)
0809 06:26:02.057786 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.057804 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:02.057924 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:02.059009 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:02.059013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:02.059039 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:02.059352 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:02.059364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.061090 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.061095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:02.063160 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:02.063175 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.064897 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.064907 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:02.065100 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.065133 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 06:26:02.065197 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 06:26:02.065330 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275}
I20260809 06:26:02.065655 20539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.057081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.8.131.3:39623)
0809 06:26:02.057764 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:02.057770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:02.057785 (+ 15us) server_negotiation.cc:407] Connection header received
0809 06:26:02.057844 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:02.057848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:02.057919 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:02.058025 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:02.058432 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.059876 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.062262 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.063478 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.065045 (+ 1567us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.065155 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:02.065210 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 06:26:02.065346 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":481,"threads_started":1}
I20260809 06:26:02.066414 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.058185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.8.131.1:41977)
0809 06:26:02.058674 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:02.058680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:02.058699 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:26:02.058768 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:02.058773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:02.058845 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:02.059628 (+ 783us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:02.059810 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.060928 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.063374 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.064627 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.065872 (+ 1245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.065973 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:02.066023 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 06:26:02.066150 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":540,"threads_started":1}
I20260809 06:26:02.066397 20317 tablet_service.cc:1467] Processing CreateTablet for tablet 6b3ecf83429a42dbad9a63e0cf810b4f (DEFAULT_TABLE table=test-table [id=fdab639fe890447bb0bc7029a803498a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:02.066876 20317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b3ecf83429a42dbad9a63e0cf810b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:02.070706 19897 tablet_service.cc:1467] Processing CreateTablet for tablet 6b3ecf83429a42dbad9a63e0cf810b4f (DEFAULT_TABLE table=test-table [id=fdab639fe890447bb0bc7029a803498a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:02.071149 19897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b3ecf83429a42dbad9a63e0cf810b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:02.072742 20545 tablet_bootstrap.cc:492] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: Bootstrap starting.
I20260809 06:26:02.074496 20545 tablet_bootstrap.cc:654] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:02.076905 20545 tablet_bootstrap.cc:492] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: No bootstrap required, opened a new log
I20260809 06:26:02.077046 20545 ts_tablet_manager.cc:1397] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 06:26:02.078060 20545 raft_consensus.cc:348] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } }
I20260809 06:26:02.078378 20545 raft_consensus.cc:374] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:02.078493 20545 raft_consensus.cc:729] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470fe37812bf43fca27d30d336572ade, State: Initialized, Role: FOLLOWER
I20260809 06:26:02.078827 20545 consensus_queue.cc:260] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } }
I20260809 06:26:02.082427 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.058589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:36541 (local address 127.0.0.1:36844)
0809 06:26:02.064708 (+ 6119us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.064727 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:02.064857 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:02.076669 (+ 11812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:02.076674 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:02.076699 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:02.076881 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:02.076901 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.078220 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.078226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:02.080323 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:02.080339 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.081667 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.081673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:02.081908 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.081944 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 06:26:02.082016 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 06:26:02.082149 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6019,"thread_start_us":11061,"threads_started":1}
I20260809 06:26:02.083288 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.058742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.8.131.2:36541)
0809 06:26:02.076367 (+ 17625us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:02.076373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:02.076401 (+ 28us) server_negotiation.cc:407] Connection header received
0809 06:26:02.076470 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:02.076473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:02.076532 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:02.076652 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:02.077128 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.078048 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.080544 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.081486 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.082691 (+ 1205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.082796 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:02.082853 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 06:26:02.082987 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17477,"thread_start_us":18030,"threads_started":1}
I20260809 06:26:02.084538 20123 tablet_service.cc:1467] Processing CreateTablet for tablet 6b3ecf83429a42dbad9a63e0cf810b4f (DEFAULT_TABLE table=test-table [id=fdab639fe890447bb0bc7029a803498a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:02.085019 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b3ecf83429a42dbad9a63e0cf810b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:02.085842 20547 tablet_bootstrap.cc:492] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835: Bootstrap starting.
I20260809 06:26:02.087725 20547 tablet_bootstrap.cc:654] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:02.088055 20545 ts_tablet_manager.cc:1428] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20260809 06:26:02.089069 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.080553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.3:34487)
0809 06:26:02.088681 (+ 8128us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.088822 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7995,"threads_started":1}
I20260809 06:26:02.089161 20419 heartbeater.cc:503] Master 127.8.131.61:33805 was elected leader, sending a full tablet report...
I20260809 06:26:02.091768 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.091212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.3:53251)
0809 06:26:02.091518 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.091598 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205}
I20260809 06:26:02.092777 20551 tablet_bootstrap.cc:492] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: Bootstrap starting.
I20260809 06:26:02.093350 20547 tablet_bootstrap.cc:492] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835: No bootstrap required, opened a new log
I20260809 06:26:02.093478 20547 ts_tablet_manager.cc:1397] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 06:26:02.094064 20547 raft_consensus.cc:348] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } }
I20260809 06:26:02.094198 20551 tablet_bootstrap.cc:654] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:02.094251 20547 raft_consensus.cc:374] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:02.094362 20547 raft_consensus.cc:729] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb66859753634ddd90ab08678d482835, State: Initialized, Role: FOLLOWER
I20260809 06:26:02.094637 20547 consensus_queue.cc:260] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } }
I20260809 06:26:02.095451 20547 ts_tablet_manager.cc:1428] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 06:26:02.096731 20551 tablet_bootstrap.cc:492] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: No bootstrap required, opened a new log
I20260809 06:26:02.096903 20551 ts_tablet_manager.cc:1397] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:26:02.097645 20000 heartbeater.cc:503] Master 127.8.131.61:33805 was elected leader, sending a full tablet report...
I20260809 06:26:02.097756 20551 raft_consensus.cc:348] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } }
I20260809 06:26:02.097991 20551 raft_consensus.cc:374] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:02.098094 20551 raft_consensus.cc:729] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b583934b073a4f10a5acf20d959e17b7, State: Initialized, Role: FOLLOWER
I20260809 06:26:02.098320 20551 consensus_queue.cc:260] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } }
I20260809 06:26:02.099728 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.095795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.1:38661)
0809 06:26:02.099266 (+ 3471us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.099412 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3315,"thread_start_us":3400,"threads_started":1}
I20260809 06:26:02.099910 20551 ts_tablet_manager.cc:1428] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 06:26:02.101114 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.100147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.2:58785)
0809 06:26:02.100740 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.100861 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":299,"threads_started":1}
I20260809 06:26:02.101768 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.100928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.1:43633)
0809 06:26:02.101515 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.101603 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"threads_started":1}
I20260809 06:26:02.102305 20216 heartbeater.cc:503] Master 127.8.131.61:33805 was elected leader, sending a full tablet report...
I20260809 06:26:02.103386 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.103066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.2:49847)
0809 06:26:02.103199 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.103274 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 06:26:02.184933 20420 tablet.cc:2369] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:26:02.293030 20217 tablet.cc:2369] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:26:02.309577 20556 raft_consensus.cc:482] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:26:02.309852 20556 raft_consensus.cc:504] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } }
I20260809 06:26:02.310595 20556 leader_election.cc:288] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623), bb66859753634ddd90ab08678d482835 (127.8.131.1:41977)
I20260809 06:26:02.311273 19926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b3ecf83429a42dbad9a63e0cf810b4f" candidate_uuid: "b583934b073a4f10a5acf20d959e17b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb66859753634ddd90ab08678d482835" is_pre_election: true
I20260809 06:26:02.311319 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b3ecf83429a42dbad9a63e0cf810b4f" candidate_uuid: "b583934b073a4f10a5acf20d959e17b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470fe37812bf43fca27d30d336572ade" is_pre_election: true
I20260809 06:26:02.311523 19926 raft_consensus.cc:2454] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b583934b073a4f10a5acf20d959e17b7 in term 0.
I20260809 06:26:02.311741 20345 raft_consensus.cc:2454] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b583934b073a4f10a5acf20d959e17b7 in term 0.
I20260809 06:26:02.312085 20058 leader_election.cc:302] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b583934b073a4f10a5acf20d959e17b7, bb66859753634ddd90ab08678d482835; no voters:
I20260809 06:26:02.312467 20556 raft_consensus.cc:2781] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:26:02.312577 20556 raft_consensus.cc:482] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:26:02.312695 20556 raft_consensus.cc:3037] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:02.314911 20556 raft_consensus.cc:504] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } }
I20260809 06:26:02.315503 20556 leader_election.cc:288] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [CANDIDATE]: Term 1 election: Requested vote from peers 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623), bb66859753634ddd90ab08678d482835 (127.8.131.1:41977)
I20260809 06:26:02.316087 19926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b3ecf83429a42dbad9a63e0cf810b4f" candidate_uuid: "b583934b073a4f10a5acf20d959e17b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb66859753634ddd90ab08678d482835"
I20260809 06:26:02.316128 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b3ecf83429a42dbad9a63e0cf810b4f" candidate_uuid: "b583934b073a4f10a5acf20d959e17b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470fe37812bf43fca27d30d336572ade"
I20260809 06:26:02.316259 19926 raft_consensus.cc:3037] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:02.316385 20345 raft_consensus.cc:3037] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:02.318356 19926 raft_consensus.cc:2454] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b583934b073a4f10a5acf20d959e17b7 in term 1.
I20260809 06:26:02.318641 20345 raft_consensus.cc:2454] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b583934b073a4f10a5acf20d959e17b7 in term 1.
I20260809 06:26:02.318846 20058 leader_election.cc:302] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b583934b073a4f10a5acf20d959e17b7, bb66859753634ddd90ab08678d482835; no voters:
I20260809 06:26:02.319152 20556 raft_consensus.cc:2781] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:02.319615 20556 raft_consensus.cc:686] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 1 LEADER]: Becoming Leader. State: Replica: b583934b073a4f10a5acf20d959e17b7, State: Running, Role: LEADER
I20260809 06:26:02.319983 20556 consensus_queue.cc:237] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } }
I20260809 06:26:02.320729 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.320297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.2:57711)
0809 06:26:02.320481 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.320583 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 06:26:02.323076 19437 catalog_manager.cc:5495] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b583934b073a4f10a5acf20d959e17b7 (127.8.131.2). New cstate: current_term: 1 leader_uuid: "b583934b073a4f10a5acf20d959e17b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b583934b073a4f10a5acf20d959e17b7" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36541 } health_report { overall_health: HEALTHY } } }
I20260809 06:26:02.372200 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.367950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36846 (local address 127.8.131.2:36541)
0809 06:26:02.368168 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:02.368173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:02.368192 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:26:02.368324 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:02.368328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:02.368410 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:02.368484 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:02.368802 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.369493 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.370676 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.371069 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.371159 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.371497 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:02.371591 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:02.371723 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:02.371821 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 06:26:02.371893 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 06:26:02.375474 20345 raft_consensus.cc:1261] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [term 1 FOLLOWER]: Refusing update from remote peer b583934b073a4f10a5acf20d959e17b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:02.375516 19926 raft_consensus.cc:1261] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [term 1 FOLLOWER]: Refusing update from remote peer b583934b073a4f10a5acf20d959e17b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:02.376154 20556 consensus_queue.cc:1035] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470fe37812bf43fca27d30d336572ade" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 39623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:02.376436 20566 consensus_queue.cc:1035] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb66859753634ddd90ab08678d482835" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:02.379710 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.378794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.1:55945)
0809 06:26:02.379143 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.379377 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260809 06:26:02.380424 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.378812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.3:49149)
0809 06:26:02.380034 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.380155 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1058}
I20260809 06:26:02.386411 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.385829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.3:33051)
0809 06:26:02.386033 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.386177 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 06:26:02.394214 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.393706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.2:53779)
0809 06:26:02.393982 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.394067 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260809 06:26:02.399427 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.398245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.2:45461)
0809 06:26:02.399092 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.399202 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":690}
I20260809 06:26:02.400343 19437 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:02.400647 19437 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:02.400971 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.400411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.8.131.1:55501)
0809 06:26:02.400674 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.400778 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260809 06:26:02.407471 19437 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52b995272cdd470b9dd9af9cffbc4879: Sending DeleteTablet for 3 replicas of tablet 6b3ecf83429a42dbad9a63e0cf810b4f
I20260809 06:26:02.408341 19897 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b3ecf83429a42dbad9a63e0cf810b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:02 UTC) from {username='oryx', principal='oryx/127.8.131.61@KRBTEST.COM'} at 127.0.0.1:54102
I20260809 06:26:02.408675 20317 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b3ecf83429a42dbad9a63e0cf810b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:02 UTC) from {username='oryx', principal='oryx/127.8.131.61@KRBTEST.COM'} at 127.0.0.1:52994
I20260809 06:26:02.409128 19896 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b3ecf83429a42dbad9a63e0cf810b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:02 UTC) from {username='oryx', principal='oryx/127.8.131.61@KRBTEST.COM'} at 127.0.0.1:54102
I20260809 06:26:02.409359 20123 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b3ecf83429a42dbad9a63e0cf810b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:02 UTC) from {username='oryx', principal='oryx/127.8.131.61@KRBTEST.COM'} at 127.0.0.1:36844
I20260809 06:26:02.410125 20585 tablet_replica.cc:330] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: stopping tablet replica
I20260809 06:26:02.410386 20585 raft_consensus.cc:2229] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:26:02.410567 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 19781
I20260809 06:26:02.410836 20585 raft_consensus.cc:2258] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:02.415194 20585 ts_tablet_manager.cc:1905] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:02.420796 20584 tablet_replica.cc:330] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: stopping tablet replica
I20260809 06:26:02.421020 20584 raft_consensus.cc:2229] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:02.421357 20584 raft_consensus.cc:2258] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:02.425065 20585 ts_tablet_manager.cc:1918] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:02.425475 20585 log.cc:1196] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/6b3ecf83429a42dbad9a63e0cf810b4f
I20260809 06:26:02.426460 20585 ts_tablet_manager.cc:1939] T 6b3ecf83429a42dbad9a63e0cf810b4f P b583934b073a4f10a5acf20d959e17b7: Deleting consensus metadata
I20260809 06:26:02.428259 19412 catalog_manager.cc:4841] TS b583934b073a4f10a5acf20d959e17b7 (127.8.131.2:36541): tablet 6b3ecf83429a42dbad9a63e0cf810b4f (table test-table [id=fdab639fe890447bb0bc7029a803498a]) successfully deleted
I20260809 06:26:02.428838 20584 ts_tablet_manager.cc:1905] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:02.429211 20583 tablet_replica.cc:330] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835: stopping tablet replica
I20260809 06:26:02.429425 20583 raft_consensus.cc:2229] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:02.429786 20583 raft_consensus.cc:2258] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:02.430891 20583 ts_tablet_manager.cc:1905] T 6b3ecf83429a42dbad9a63e0cf810b4f P bb66859753634ddd90ab08678d482835: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:02.431783 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.412694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.8.131.1:41977)
0809 06:26:02.412993 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:02.413002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:02.413026 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:26:02.413091 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:02.413099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:02.413215 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:02.413339 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:02.415975 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.417406 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.419849 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.420720 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.420831 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.424345 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:02.424358 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:02.424374 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:02.424412 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:02.427707 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:02.428904 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:02.428920 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:02.428923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:02.429084 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:02.429593 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:02.429600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:02.429608 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:02.430157 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:02.430538 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:02.431177 (+ 639us) server_negotiation.cc:299] Negotiation successful
0809 06:26:02.431415 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:26:02.432785 19991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:02.436124 20584 ts_tablet_manager.cc:1918] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:02.436273 20584 log.cc:1196] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/6b3ecf83429a42dbad9a63e0cf810b4f
I20260809 06:26:02.436793 20584 ts_tablet_manager.cc:1939] T 6b3ecf83429a42dbad9a63e0cf810b4f P 470fe37812bf43fca27d30d336572ade: Deleting consensus metadata
I20260809 06:26:02.437948 19411 catalog_manager.cc:4841] TS 470fe37812bf43fca27d30d336572ade (127.8.131.3:39623): tablet 6b3ecf83429a42dbad9a63e0cf810b4f (table test-table [id=fdab639fe890447bb0bc7029a803498a]) successfully deleted
I20260809 06:26:02.529404 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 19781
I20260809 06:26:02.554527 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 20017
I20260809 06:26:02.556169 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.555777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54120)
0809 06:26:02.555935 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.556039 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 06:26:02.556429 19412 catalog_manager.cc:4584] TS bb66859753634ddd90ab08678d482835 (127.8.131.1:41977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b3ecf83429a42dbad9a63e0cf810b4f: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
I20260809 06:26:02.568383 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.555408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.8.131.2:36541)
0809 06:26:02.555593 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:02.555601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:02.555625 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:26:02.555872 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:02.555877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:02.555951 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:02.556062 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:02.557493 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.558201 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.559467 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.560035 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.560180 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.562721 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:02.562747 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:02.562763 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:02.562796 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:02.565585 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:02.566257 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:02.566265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:02.566268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:02.566403 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:02.566791 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:02.566797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:02.566800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:02.567176 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:02.567496 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:02.567814 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 06:26:02.568060 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:26:02.569054 20203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:02.609050 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.608525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54130)
0809 06:26:02.608789 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.608886 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 06:26:02.610284 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.609851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54138)
0809 06:26:02.610033 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.610113 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 06:26:02.617455 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.617069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54152)
0809 06:26:02.617239 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.617335 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 06:26:02.618428 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.618087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54162)
0809 06:26:02.618235 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.618299 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 06:26:02.650158 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.649711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33182)
0809 06:26:02.649906 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.650004 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 06:26:02.651213 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.650850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45687 (local address 127.0.0.1:33184)
0809 06:26:02.651010 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.651092 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:45687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 06:26:02.679998 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.679477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54166)
0809 06:26:02.679737 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.679833 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 06:26:02.681162 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.680717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54170)
0809 06:26:02.680903 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.680989 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 06:26:02.694653 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.694147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54182)
0809 06:26:02.694365 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.694509 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 06:26:02.695250 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 20017
I20260809 06:26:02.696195 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.695446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54186)
0809 06:26:02.695703 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.695998 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 06:26:02.722007 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 20229
I20260809 06:26:02.736366 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.722803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.8.131.3:39623)
0809 06:26:02.723346 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:02.723351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:02.723366 (+ 15us) server_negotiation.cc:407] Connection header received
0809 06:26:02.723420 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:02.723424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:02.723500 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:02.723622 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:02.725181 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.725981 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.727402 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:02.727910 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:02.728073 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:02.730639 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:02.730653 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:02.730669 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:02.730703 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:02.733413 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:02.734119 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:02.734126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:02.734128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:02.734267 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:02.734660 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:02.734666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:02.734671 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:02.735045 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:02.735389 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:02.735784 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 06:26:02.736018 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":273,"threads_started":1}
I20260809 06:26:02.737074 20406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:02.783246 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.782755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54196)
0809 06:26:02.782974 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.783102 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 06:26:02.784420 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.784023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54198)
0809 06:26:02.784177 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.784257 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 06:26:02.795588 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.795188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54214)
0809 06:26:02.795377 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.795452 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 06:26:02.796725 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:02.796369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41977 (local address 127.0.0.1:54220)
0809 06:26:02.796526 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:02.796590 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 06:26:02.833168 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 20229
I20260809 06:26:02.858764 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 19223
I20260809 06:26:02.860052 19478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:02.986362 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 19223
I20260809 06:26:03.011371 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 19378
I20260809 06:26:03.012497 19778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:03.130115 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 19378
2026-08-09T06:26:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10092 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:26:03 dist-test-slave-mjjr krb5kdc[20601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:26:03 dist-test-slave-mjjr krb5kdc[20601](info): set up 2 sockets
Aug 09 06:26:03 dist-test-slave-mjjr krb5kdc[20601](info): commencing operation
krb5kdc: starting...
W20260809 06:26:04.453112 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:26:04 dist-test-slave-mjjr krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:26:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:26:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.62.keytab.
Entry for principal oryx/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.62.keytab.
I20260809 06:26:04.715116 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:40179
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.62.keytab
--principal=oryx/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:40179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:05.057118 20619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:05.057543 20619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:05.057729 20619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:05.065367 20619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:26:05.065570 20619 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 06:26:05.065678 20619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:05.065832 20619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:05.065932 20619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:26:05.066049 20619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:26:05.076624 20619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:40179
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.62.keytab
--principal=oryx/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:40179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:05.078087 20619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:05.079531 20619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:05.092434 20624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:05.092360 20625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:05.092473 20627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:05.093741 20619 server_base.cc:1034] running on GCE node
I20260809 06:26:05.094588 20619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:05.096042 20619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:05.097294 20619 hybrid_clock.cc:648] HybridClock initialized: now 1786256765097216 us; error 56 us; skew 500 ppm
I20260809 06:26:05.097627 20619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:05 dist-test-slave-mjjr krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256765, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:05.100852 20619 init.cc:377] Logged in from keytab as oryx/127.8.131.62@KRBTEST.COM (short username oryx)
I20260809 06:26:05.102340 20619 webserver.cc:460] Webserver started at http://127.8.131.62:37177/ using document root <none> and password file <none>
I20260809 06:26:05.103060 20619 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:05.103205 20619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:05.103543 20619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:05.106700 20619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "61c2e9b2009c482e8684c44ec3602564"
format_stamp: "Formatted at 2026-08-09 06:26:05 on dist-test-slave-mjjr"
server_key: "11717db2c8f9f545aa06226c8f9d17f1"
server_key_iv: "cbda927a8388831c52299728820550b7"
server_key_version: "encryptionkey@0"
I20260809 06:26:05.107576 20619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "61c2e9b2009c482e8684c44ec3602564"
format_stamp: "Formatted at 2026-08-09 06:26:05 on dist-test-slave-mjjr"
server_key: "11717db2c8f9f545aa06226c8f9d17f1"
server_key_iv: "cbda927a8388831c52299728820550b7"
server_key_version: "encryptionkey@0"
I20260809 06:26:05.112898 20619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:26:05.116459 20639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:05.117761 20619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:26:05.117982 20619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "61c2e9b2009c482e8684c44ec3602564"
format_stamp: "Formatted at 2026-08-09 06:26:05 on dist-test-slave-mjjr"
server_key: "11717db2c8f9f545aa06226c8f9d17f1"
server_key_iv: "cbda927a8388831c52299728820550b7"
server_key_version: "encryptionkey@0"
I20260809 06:26:05.118209 20619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:05.157299 20619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:05.161964 20619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:05.162251 20619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:05.185070 20619 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:40179
I20260809 06:26:05.185089 20729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:40179 every 8 connection(s)
I20260809 06:26:05.186537 20619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:26:05.190369 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 20619
I20260809 06:26:05.190616 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:26:05.190675 20730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:05.191506 8716 external_mini_cluster.cc:1428] Setting key 3b5b5798e2d3df6f802c0846a5b73ddb
I20260809 06:26:05.204624 20730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564: Bootstrap starting.
Aug 09 06:26:05 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:26:05.208398 20730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:05.209622 20730 log.cc:824] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:05.212682 20730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564: No bootstrap required, opened a new log
I20260809 06:26:05.216225 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:05.193628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55840 (local address 127.8.131.62:40179)
0809 06:26:05.194506 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:05.194518 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:05.194556 (+ 38us) server_negotiation.cc:407] Connection header received
0809 06:26:05.196933 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:05.196954 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:05.197359 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:05.197681 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:05.199448 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.200841 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.202649 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.203325 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.203468 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:05.208326 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:05.208335 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:05.208349 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:05.208393 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:05.212030 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:05.212752 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:05.212763 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:05.212799 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:05.213020 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:05.213408 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:05.213414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:05.213417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:05.213944 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:05.214286 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:05.214741 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 06:26:05.215169 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":432,"threads_started":1}
I20260809 06:26:05.220595 20730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c2e9b2009c482e8684c44ec3602564" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40179 } }
I20260809 06:26:05.220961 20730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:05.221078 20730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c2e9b2009c482e8684c44ec3602564, State: Initialized, Role: FOLLOWER
I20260809 06:26:05.221709 20730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c2e9b2009c482e8684c44ec3602564" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40179 } }
I20260809 06:26:05.221977 20730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:26:05.222136 20730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:26:05.222327 20730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:05.224839 20730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c2e9b2009c482e8684c44ec3602564" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40179 } }
I20260809 06:26:05.225333 20730 leader_election.cc:302] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61c2e9b2009c482e8684c44ec3602564; no voters:
I20260809 06:26:05.225886 20730 leader_election.cc:288] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:26:05.226016 20740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:05.226786 20740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [term 1 LEADER]: Becoming Leader. State: Replica: 61c2e9b2009c482e8684c44ec3602564, State: Running, Role: LEADER
I20260809 06:26:05.227267 20740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c2e9b2009c482e8684c44ec3602564" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40179 } }
I20260809 06:26:05.228058 20730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:26:05.229946 20742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61c2e9b2009c482e8684c44ec3602564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c2e9b2009c482e8684c44ec3602564" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40179 } } }
I20260809 06:26:05.230093 20743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61c2e9b2009c482e8684c44ec3602564. Latest consensus state: current_term: 1 leader_uuid: "61c2e9b2009c482e8684c44ec3602564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c2e9b2009c482e8684c44ec3602564" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40179 } } }
I20260809 06:26:05.230640 20742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:05.230644 20743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564 [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:05.233220 20747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:26:05.238852 20747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:26:05.249260 20747 catalog_manager.cc:1347] Generated new cluster ID: 08c8111a37d04a31a6c9b7c1f5cafd65
I20260809 06:26:05.249382 20747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:26:05.270128 20747 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:26:05.271620 20747 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:26:05.280114 20747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564: Generated new TSK 0
I20260809 06:26:05.281162 20747 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab.
Entry for principal oryx/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab.
I20260809 06:26:05.351768 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:40179
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab
--principal=oryx/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:05.698611 20767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:05.698902 20767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:05.699002 20767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:05.707419 20767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:05.707537 20767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:05.707625 20767 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 06:26:05.707794 20767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:26:05.717949 20767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab
--principal=oryx/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.131.62:40179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:05.719242 20767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:05.720602 20767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:05.734076 20778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:05.734045 20783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:05.734110 20780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:05.735229 20767 server_base.cc:1034] running on GCE node
I20260809 06:26:05.735986 20767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:05.737035 20767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:05.738292 20767 hybrid_clock.cc:648] HybridClock initialized: now 1786256765738223 us; error 40 us; skew 500 ppm
I20260809 06:26:05.738591 20767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:05 dist-test-slave-mjjr krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256765, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:05.741937 20767 init.cc:377] Logged in from keytab as oryx/127.8.131.1@KRBTEST.COM (short username oryx)
I20260809 06:26:05.743435 20767 webserver.cc:460] Webserver started at http://127.8.131.1:44003/ using document root <none> and password file <none>
I20260809 06:26:05.744196 20767 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:05.744315 20767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:05.744659 20767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:05.747632 20767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "b047aac8e3714d96a76439e4139b6726"
format_stamp: "Formatted at 2026-08-09 06:26:05 on dist-test-slave-mjjr"
server_key: "847c4d725aa4d6b90a053ccb2f0110ce"
server_key_iv: "2e152d74f325ba6bdbbde30e8663b52a"
server_key_version: "encryptionkey@0"
I20260809 06:26:05.748536 20767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "b047aac8e3714d96a76439e4139b6726"
format_stamp: "Formatted at 2026-08-09 06:26:05 on dist-test-slave-mjjr"
server_key: "847c4d725aa4d6b90a053ccb2f0110ce"
server_key_iv: "2e152d74f325ba6bdbbde30e8663b52a"
server_key_version: "encryptionkey@0"
I20260809 06:26:05.753657 20767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 06:26:05.757174 20793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:05.758424 20767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:26:05.758599 20767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "b047aac8e3714d96a76439e4139b6726"
format_stamp: "Formatted at 2026-08-09 06:26:05 on dist-test-slave-mjjr"
server_key: "847c4d725aa4d6b90a053ccb2f0110ce"
server_key_iv: "2e152d74f325ba6bdbbde30e8663b52a"
server_key_version: "encryptionkey@0"
I20260809 06:26:05.758824 20767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:05.780488 20767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:05.784890 20767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:05.785182 20767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:05.788153 20767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:05.788269 20767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:05.788440 20767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:05.788558 20767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 09 06:26:05 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256765, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:26:05.813898 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:05.788765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:44725 (local address 127.8.131.62:40179)
0809 06:26:05.789543 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:05.789548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:05.790351 (+ 803us) server_negotiation.cc:407] Connection header received
0809 06:26:05.792467 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:05.792473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:05.792563 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:05.792695 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:05.795023 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.796137 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.799587 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.800080 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.800186 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:05.806237 (+ 6051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:05.806242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:05.806245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:05.806280 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:05.810132 (+ 3852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:05.811024 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:05.811031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:05.811035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:05.811168 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:05.811628 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:05.811633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:05.811637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:05.811967 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:05.812146 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:05.813352 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0809 06:26:05.813632 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":547,"threads_started":1}
I20260809 06:26:05.815974 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:05.789323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40179 (local address 127.8.131.1:44725)
0809 06:26:05.790187 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:05.790233 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:05.792178 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:05.792952 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:05.792960 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:05.793738 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:05.794728 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:05.794767 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.796373 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.796380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:05.799389 (+ 3009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:05.799408 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.800365 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.800375 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:05.800784 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:05.801664 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:05.801711 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:05.805958 (+ 4247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:05.810318 (+ 4360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:05.810332 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:05.810363 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:05.810866 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:05.811292 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:05.811297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:05.811301 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:05.811506 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:05.812169 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:05.812177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:05.812806 (+ 629us) client_negotiation.cc:769] Sending connection context
0809 06:26:05.813217 (+ 411us) client_negotiation.cc:240] Negotiation successful
0809 06:26:05.813971 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":388,"threads_started":1}
I20260809 06:26:05.836977 20767 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:35815
I20260809 06:26:05.836982 20968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:35815 every 8 connection(s)
I20260809 06:26:05.838562 20767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:26:05.839743 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:05.823694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:46101 (local address 127.8.131.62:40179)
0809 06:26:05.823884 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:05.823888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:05.824786 (+ 898us) server_negotiation.cc:407] Connection header received
0809 06:26:05.825001 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:05.825009 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:05.825125 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:05.825245 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:05.826888 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.827876 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.829283 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.830149 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.830304 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:05.833012 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:05.833019 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:05.833024 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:05.833067 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:05.836940 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:05.837712 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:05.837717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:05.837721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:05.837885 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:05.838244 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:05.838251 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:05.838255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:05.838546 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:05.838700 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:05.839115 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 06:26:05.839345 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:26:05.839843 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:05.823853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40179 (local address 127.8.131.1:46101)
0809 06:26:05.824667 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:05.824688 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:05.824854 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:05.825304 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:05.825312 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:05.826255 (+ 943us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:05.826701 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:05.826719 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.828064 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.828068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:05.829117 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:05.829128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:05.830373 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:05.830378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:05.830504 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:05.831216 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:05.831246 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:05.832750 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:05.837098 (+ 4348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:05.837106 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:05.837112 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:05.837589 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:05.838010 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:05.838013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:05.838015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:05.838131 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:05.838667 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:05.838671 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:05.838865 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 06:26:05.839022 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 06:26:05.839355 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":547,"threads_started":1}
I20260809 06:26:05.841835 20969 heartbeater.cc:347] Connected to a master server at 127.8.131.62:40179
I20260809 06:26:05.842072 20969 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:05.842801 20969 heartbeater.cc:511] Master 127.8.131.62:40179 requested a full tablet report, sending...
I20260809 06:26:05.844952 20664 ts_manager.cc:194] Registered new tserver with Master: b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815)
I20260809 06:26:05.846454 20664 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:44725
I20260809 06:26:05.848737 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 20767
I20260809 06:26:05.848959 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:26:05.849637 8716 external_mini_cluster.cc:1428] Setting key ae566758708efc93202f16e1052b3ae4
WARNING: no policy specified for oryx/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab.
Entry for principal oryx/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab.
I20260809 06:26:05.906388 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:40179
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab
--principal=oryx/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:06.237290 20973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:06.237600 20973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:06.237795 20973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:06.245218 20973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:06.245332 20973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:06.245507 20973 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 06:26:06.245644 20973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:26:06.256117 20973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab
--principal=oryx/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.131.62:40179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:06.257427 20973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:06.258790 20973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:06.272382 20980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:06.272496 20984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:06.272445 20981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:06.273439 20973 server_base.cc:1034] running on GCE node
I20260809 06:26:06.274062 20973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:06.274971 20973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:06.276211 20973 hybrid_clock.cc:648] HybridClock initialized: now 1786256766276132 us; error 54 us; skew 500 ppm
I20260809 06:26:06.276513 20973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:06 dist-test-slave-mjjr krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256766, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:06.279748 20973 init.cc:377] Logged in from keytab as oryx/127.8.131.2@KRBTEST.COM (short username oryx)
I20260809 06:26:06.281293 20973 webserver.cc:460] Webserver started at http://127.8.131.2:44961/ using document root <none> and password file <none>
I20260809 06:26:06.281998 20973 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:06.282132 20973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:06.282470 20973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:06.285444 20973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e"
format_stamp: "Formatted at 2026-08-09 06:26:06 on dist-test-slave-mjjr"
server_key: "2095d2406245ffb67668e5738b923d28"
server_key_iv: "91dccb25ef92020a33f5eac52c2055f7"
server_key_version: "encryptionkey@0"
I20260809 06:26:06.286305 20973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e"
format_stamp: "Formatted at 2026-08-09 06:26:06 on dist-test-slave-mjjr"
server_key: "2095d2406245ffb67668e5738b923d28"
server_key_iv: "91dccb25ef92020a33f5eac52c2055f7"
server_key_version: "encryptionkey@0"
I20260809 06:26:06.291524 20973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:26:06.294881 20994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:06.296237 20973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:26:06.296418 20973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e"
format_stamp: "Formatted at 2026-08-09 06:26:06 on dist-test-slave-mjjr"
server_key: "2095d2406245ffb67668e5738b923d28"
server_key_iv: "91dccb25ef92020a33f5eac52c2055f7"
server_key_version: "encryptionkey@0"
I20260809 06:26:06.296639 20973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:06.319747 20973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:06.324079 20973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:06.324373 20973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:06.327069 20973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:06.327179 20973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:06.327335 20973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:06.327412 20973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 06:26:06 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256766, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:26:06.353339 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:06.327801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:48603 (local address 127.8.131.62:40179)
0809 06:26:06.328018 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:06.328026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:06.329762 (+ 1736us) server_negotiation.cc:407] Connection header received
0809 06:26:06.332292 (+ 2530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:06.332299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:06.332393 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:06.332492 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:06.335027 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.336251 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.339613 (+ 3362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.340328 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.340476 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:06.345399 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:06.345407 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:06.345416 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:06.345451 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:06.348992 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:06.350052 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:06.350055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:06.350058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:06.350233 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:06.350877 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:06.350887 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:06.350892 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:06.351354 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:06.351542 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:06.352491 (+ 949us) server_negotiation.cc:299] Negotiation successful
0809 06:26:06.352791 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 06:26:06.354135 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:06.328618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40179 (local address 127.8.131.2:48603)
0809 06:26:06.329597 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:06.329642 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:06.331912 (+ 2270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:06.333043 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:06.333059 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:06.333900 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:06.334819 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:06.334848 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.336473 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.336485 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:06.339425 (+ 2940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:06.339448 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.340629 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.340640 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:06.340849 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:06.341779 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:06.341834 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:06.345179 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:06.349184 (+ 4005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:06.349197 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:06.349216 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:06.349841 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:06.350369 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:06.350374 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:06.350377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:06.350637 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:06.351546 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:06.351555 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:06.351981 (+ 426us) client_negotiation.cc:769] Sending connection context
0809 06:26:06.352364 (+ 383us) client_negotiation.cc:240] Negotiation successful
0809 06:26:06.352952 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":375,"threads_started":1}
I20260809 06:26:06.376227 20973 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:34645
I20260809 06:26:06.376251 21167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:34645 every 8 connection(s)
I20260809 06:26:06.377254 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:06.361080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:60515 (local address 127.8.131.62:40179)
0809 06:26:06.361255 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:06.361263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:06.361286 (+ 23us) server_negotiation.cc:407] Connection header received
0809 06:26:06.361543 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:06.361549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:06.361657 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:06.361767 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:06.363295 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.364322 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.367056 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.367442 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.367537 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:06.370405 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:06.370413 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:06.370421 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:06.370467 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:06.374206 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:06.375081 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:06.375087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:06.375090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:06.375240 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:06.375676 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:06.375681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:06.375685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:06.376018 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:06.376178 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:06.376681 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 06:26:06.376912 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:26:06.377395 21110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:06.360505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40179 (local address 127.8.131.2:60515)
0809 06:26:06.361213 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:06.361232 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:06.361366 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:06.361837 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:06.361848 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:06.362609 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:06.363092 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:06.363109 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.364503 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.364511 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:06.366895 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:06.366913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.367667 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.367674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:06.367871 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:06.368624 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:06.368657 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:06.370142 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:06.374327 (+ 4185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:06.374332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:06.374335 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:06.374944 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:06.375394 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:06.375398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:06.375401 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:06.375524 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:06.376149 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:06.376154 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:06.376372 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 06:26:06.376589 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 06:26:06.376931 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":375,"threads_started":1}
I20260809 06:26:06.377718 20973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:26:06.380074 21170 heartbeater.cc:347] Connected to a master server at 127.8.131.62:40179
I20260809 06:26:06.380350 21170 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:06.381054 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 20973
I20260809 06:26:06.381237 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:26:06.381291 21170 heartbeater.cc:511] Master 127.8.131.62:40179 requested a full tablet report, sending...
I20260809 06:26:06.381660 8716 external_mini_cluster.cc:1428] Setting key 0abff86a486fd59c5c42cf59a1b81702
I20260809 06:26:06.382756 20664 ts_manager.cc:194] Registered new tserver with Master: 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645)
I20260809 06:26:06.383598 20664 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:48603
WARNING: no policy specified for oryx/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab.
Entry for principal oryx/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab.
I20260809 06:26:06.434877 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:40179
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab
--principal=oryx/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:06.775781 21177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:06.776082 21177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:06.776188 21177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:06.783660 21177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:06.783794 21177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:06.783895 21177 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 06:26:06.784016 21177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:26:06.794287 21177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab
--principal=oryx/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.131.62:40179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:06.795578 21177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:06.796990 21177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:06.811082 21192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:06.811118 21189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:06.811182 21185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:06.812055 21177 server_base.cc:1034] running on GCE node
I20260809 06:26:06.812704 21177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:06.813656 21177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:06.814932 21177 hybrid_clock.cc:648] HybridClock initialized: now 1786256766814831 us; error 60 us; skew 500 ppm
I20260809 06:26:06.815255 21177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:06 dist-test-slave-mjjr krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256766, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:06.818688 21177 init.cc:377] Logged in from keytab as oryx/127.8.131.3@KRBTEST.COM (short username oryx)
I20260809 06:26:06.820276 21177 webserver.cc:460] Webserver started at http://127.8.131.3:33967/ using document root <none> and password file <none>
I20260809 06:26:06.820996 21177 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:06.821132 21177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:06.821540 21177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:06.824663 21177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "194b90f12683478ba4a0e39cc1bc6d96"
format_stamp: "Formatted at 2026-08-09 06:26:06 on dist-test-slave-mjjr"
server_key: "4305f5b9b79ac080b3003be4eec3f425"
server_key_iv: "69459104dd74b706919a3a933e2a1640"
server_key_version: "encryptionkey@0"
I20260809 06:26:06.825485 21177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "194b90f12683478ba4a0e39cc1bc6d96"
format_stamp: "Formatted at 2026-08-09 06:26:06 on dist-test-slave-mjjr"
server_key: "4305f5b9b79ac080b3003be4eec3f425"
server_key_iv: "69459104dd74b706919a3a933e2a1640"
server_key_version: "encryptionkey@0"
I20260809 06:26:06.830879 21177 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:26:06.834452 21202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:06.835901 21177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:26:06.836072 21177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "194b90f12683478ba4a0e39cc1bc6d96"
format_stamp: "Formatted at 2026-08-09 06:26:06 on dist-test-slave-mjjr"
server_key: "4305f5b9b79ac080b3003be4eec3f425"
server_key_iv: "69459104dd74b706919a3a933e2a1640"
server_key_version: "encryptionkey@0"
I20260809 06:26:06.836320 21177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:06.849234 20969 heartbeater.cc:503] Master 127.8.131.62:40179 was elected leader, sending a full tablet report...
I20260809 06:26:06.856281 21177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:06.860844 21177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:06.861153 21177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:06.864128 21177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:06.864274 21177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:06.864459 21177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:06.864537 21177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 06:26:06 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256766, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:26:06.892768 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:06.865138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:51619 (local address 127.8.131.62:40179)
0809 06:26:06.865418 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:06.865428 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:06.866662 (+ 1234us) server_negotiation.cc:407] Connection header received
0809 06:26:06.869042 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:06.869048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:06.869156 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:06.869296 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:06.871706 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.872803 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.876134 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.876833 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.877005 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:06.882726 (+ 5721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:06.882733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:06.882741 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:06.882798 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:06.886564 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:06.887786 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:06.887790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:06.887793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:06.887986 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:06.888740 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:06.888745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:06.888749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:06.889247 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:06.889454 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:06.891934 (+ 2480us) server_negotiation.cc:299] Negotiation successful
0809 06:26:06.892250 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 06:26:06.893694 21220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:06.865482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40179 (local address 127.8.131.3:51619)
0809 06:26:06.866455 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:06.866520 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:06.868719 (+ 2199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:06.869555 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:06.869563 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:06.870432 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:06.871459 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:06.871494 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.872998 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.873011 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:06.875971 (+ 2960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:06.875986 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.877359 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.877365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:06.877738 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:06.878680 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:06.878723 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:06.882459 (+ 3736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:06.886821 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:06.886834 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:06.886859 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:06.887558 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:06.888159 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:06.888173 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:06.888179 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:06.888542 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:06.889565 (+ 1023us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:06.889579 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:06.890226 (+ 647us) client_negotiation.cc:769] Sending connection context
0809 06:26:06.890648 (+ 422us) client_negotiation.cc:240] Negotiation successful
0809 06:26:06.891437 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":432,"threads_started":1}
I20260809 06:26:06.919040 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:06.902905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:39799 (local address 127.8.131.62:40179)
0809 06:26:06.903096 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:06.903106 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:06.903925 (+ 819us) server_negotiation.cc:407] Connection header received
0809 06:26:06.904161 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:06.904168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:06.904274 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:06.904400 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:06.905465 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.906599 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.908507 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.909201 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.909357 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:06.912788 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:06.912802 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:06.912806 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:06.912843 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:06.915767 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:06.916758 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:06.916766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:06.916771 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:06.916967 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:06.917454 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:06.917459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:06.917462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:06.917793 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:06.917943 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:06.918382 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 06:26:06.918652 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:26:06.919296 21342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:06.902942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40179 (local address 127.8.131.3:39799)
0809 06:26:06.903777 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:06.903806 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:06.904006 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:06.904439 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:06.904444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:06.904961 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:06.905294 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:06.905305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.906783 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.906788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:06.908248 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:06.908262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.909800 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.909810 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:06.910029 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:06.910970 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:06.910989 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:06.912541 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:06.915911 (+ 3370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:06.915920 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:06.915924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:06.916618 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:06.917119 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:06.917124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:06.917128 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:06.917294 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:06.917921 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:06.917925 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:06.918118 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 06:26:06.918262 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 06:26:06.918655 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":428,"threads_started":1}
I20260809 06:26:06.921191 21177 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:33689
I20260809 06:26:06.921234 21381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:33689 every 8 connection(s)
I20260809 06:26:06.922675 21177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:26:06.924926 21383 heartbeater.cc:347] Connected to a master server at 127.8.131.62:40179
I20260809 06:26:06.925164 21383 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:06.925805 21383 heartbeater.cc:511] Master 127.8.131.62:40179 requested a full tablet report, sending...
I20260809 06:26:06.927126 20664 ts_manager.cc:194] Registered new tserver with Master: 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3:33689)
I20260809 06:26:06.928040 20664 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:51619
I20260809 06:26:06.930286 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 21177
I20260809 06:26:06.930505 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:26:06.930977 8716 external_mini_cluster.cc:1428] Setting key 692fdf939db0eaaa992a11cec4e9de0f
I20260809 06:26:06.934015 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 06:26:06 dist-test-slave-mjjr krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256766, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 06:26:06 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:06 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 06:26:06.991554 20805 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:06.980755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39532 (local address 127.8.131.62:40179)
0809 06:26:06.980978 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:06.980985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:06.981199 (+ 214us) server_negotiation.cc:407] Connection header received
0809 06:26:06.981347 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:06.981351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:06.981418 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:06.981513 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:06.982836 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.983557 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.984817 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.985359 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.985493 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:06.991197 (+ 5704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39532: BlockingRecv error: recv got EOF from 127.0.0.1:39532 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 06:26:06 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256766, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:26:07.008059 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:06.994394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39538 (local address 127.8.131.62:40179)
0809 06:26:06.994642 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:06.994647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:06.994848 (+ 201us) server_negotiation.cc:407] Connection header received
0809 06:26:06.994974 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:06.994978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:06.995051 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:06.995132 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:06.996415 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.997146 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.998418 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:06.998782 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:06.998871 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.002560 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:07.002565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:07.002568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:07.002602 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:07.005549 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:07.006307 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:07.006311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:07.006313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:07.006436 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:07.006793 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:07.006796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:07.006798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:07.007067 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:07.007290 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.007541 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.007776 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 06:26:07.012172 20664 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:26:07.019104 20664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:26:07.056448 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.031619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.8.131.1:35815)
0809 06:26:07.047485 (+ 15866us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.047492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.047516 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:26:07.047593 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.047599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.047904 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.048105 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:07.049893 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.051580 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.053727 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.055011 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.055145 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.055411 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.055959 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.056132 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15709,"thread_start_us":15548,"threads_started":1}
I20260809 06:26:07.056553 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.031532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35815 (local address 127.0.0.1:45150)
0809 06:26:07.046450 (+ 14918us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.046529 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.046809 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.049255 (+ 2446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.049266 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.049306 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:07.049716 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.049726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.051791 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.051795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.053559 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.053571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.055515 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.055519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.055697 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.055751 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.055891 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.056089 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14469,"mutex_wait_us":6,"spinlock_wait_cycles":21120,"thread_start_us":16156,"threads_started":1}
I20260809 06:26:07.057320 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.031898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:33689 (local address 127.0.0.1:35088)
0809 06:26:07.046523 (+ 14625us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.046549 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.046809 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.048388 (+ 1579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.048394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.048425 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:07.048964 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.048983 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.051786 (+ 2803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.051792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.054285 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.054305 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.056264 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.056271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.056440 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.056474 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.056649 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.056777 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14421,"mutex_wait_us":63,"spinlock_wait_cycles":6528,"thread_start_us":15147,"threads_started":1}
I20260809 06:26:07.057829 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.048815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:34645 (local address 127.0.0.1:56458)
0809 06:26:07.049479 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.049499 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.049624 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.050204 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.050209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.050233 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:07.050691 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.050703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.052555 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.052607 (+ 52us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.054794 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.054808 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.056936 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.056941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.057146 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.057186 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.057285 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.057455 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":330,"threads_started":1}
I20260809 06:26:07.058257 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.032435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35088 (local address 127.8.131.3:33689)
0809 06:26:07.047709 (+ 15274us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.047717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.047749 (+ 32us) server_negotiation.cc:407] Connection header received
0809 06:26:07.047837 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.047844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.048080 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.048270 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:07.049963 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.051611 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.054498 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.056023 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.057352 (+ 1329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.057563 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.057628 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.057784 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15070,"thread_start_us":16088,"threads_started":1}
I20260809 06:26:07.058588 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.048375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.8.131.2:34645)
0809 06:26:07.049696 (+ 1321us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.049706 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.049732 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:26:07.049846 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.049853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.050030 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.050212 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:07.050900 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.052367 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.054978 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.056455 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.056582 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.056779 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.058078 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.058262 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1158,"thread_start_us":1024,"threads_started":1}
I20260809 06:26:07.061096 20873 tablet_service.cc:1467] Processing CreateTablet for tablet b1011769765a4ae0aac0fd453b073fb8 (DEFAULT_TABLE table=test-table [id=44e5602f56f74cb2a692dea253853166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:07.061540 21071 tablet_service.cc:1467] Processing CreateTablet for tablet b1011769765a4ae0aac0fd453b073fb8 (DEFAULT_TABLE table=test-table [id=44e5602f56f74cb2a692dea253853166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:07.061708 21279 tablet_service.cc:1467] Processing CreateTablet for tablet b1011769765a4ae0aac0fd453b073fb8 (DEFAULT_TABLE table=test-table [id=44e5602f56f74cb2a692dea253853166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:07.062716 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1011769765a4ae0aac0fd453b073fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:07.062847 21071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1011769765a4ae0aac0fd453b073fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:07.063051 21279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1011769765a4ae0aac0fd453b073fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:07.074055 21414 tablet_bootstrap.cc:492] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Bootstrap starting.
I20260809 06:26:07.077167 21417 tablet_bootstrap.cc:492] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: Bootstrap starting.
I20260809 06:26:07.078639 21414 tablet_bootstrap.cc:654] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:07.079999 21414 log.cc:824] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:07.080468 21418 tablet_bootstrap.cc:492] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: Bootstrap starting.
I20260809 06:26:07.080914 21417 tablet_bootstrap.cc:654] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:07.082485 21417 log.cc:824] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:07.083359 21418 tablet_bootstrap.cc:654] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:07.084632 21418 log.cc:824] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:07.085570 21417 tablet_bootstrap.cc:492] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: No bootstrap required, opened a new log
I20260809 06:26:07.085754 21414 tablet_bootstrap.cc:492] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: No bootstrap required, opened a new log
I20260809 06:26:07.085861 21417 ts_tablet_manager.cc:1397] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 06:26:07.086050 21414 ts_tablet_manager.cc:1397] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.009s
I20260809 06:26:07.087745 21418 tablet_bootstrap.cc:492] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: No bootstrap required, opened a new log
I20260809 06:26:07.088003 21418 ts_tablet_manager.cc:1397] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260809 06:26:07.094075 21417 raft_consensus.cc:348] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.094439 21417 raft_consensus.cc:374] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:07.094563 21417 raft_consensus.cc:729] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b047aac8e3714d96a76439e4139b6726, State: Initialized, Role: FOLLOWER
I20260809 06:26:07.095168 21417 consensus_queue.cc:260] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.096735 21414 raft_consensus.cc:348] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.097154 21414 raft_consensus.cc:374] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:07.097251 21414 raft_consensus.cc:729] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8226868cd3dc40f19e8b3cb9cf5d4a3e, State: Initialized, Role: FOLLOWER
I20260809 06:26:07.097831 21414 consensus_queue.cc:260] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.098560 21418 raft_consensus.cc:348] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.098948 21417 ts_tablet_manager.cc:1428] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: Time spent starting tablet: real 0.013s user 0.001s sys 0.009s
I20260809 06:26:07.099085 21418 raft_consensus.cc:374] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:07.099200 21418 raft_consensus.cc:729] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194b90f12683478ba4a0e39cc1bc6d96, State: Initialized, Role: FOLLOWER
I20260809 06:26:07.099854 21170 heartbeater.cc:503] Master 127.8.131.62:40179 was elected leader, sending a full tablet report...
I20260809 06:26:07.100517 21414 ts_tablet_manager.cc:1428] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent starting tablet: real 0.014s user 0.006s sys 0.011s
I20260809 06:26:07.100883 21418 consensus_queue.cc:260] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.102666 21418 ts_tablet_manager.cc:1428] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260809 06:26:07.103106 21383 heartbeater.cc:503] Master 127.8.131.62:40179 was elected leader, sending a full tablet report...
W20260809 06:26:07.130013 21172 tablet.cc:2369] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:26:07.174636 21384 tablet.cc:2369] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:26:07.294538 21424 raft_consensus.cc:482] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:26:07.294880 21424 raft_consensus.cc:504] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.296847 21424 leader_election.cc:288] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815), 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645)
I20260809 06:26:07.303409 21220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.297267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35815 (local address 127.8.131.3:37669)
0809 06:26:07.297496 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.297514 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.297677 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.298148 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.298153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.298184 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:07.298432 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.298447 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.299746 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.299749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.301640 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.301655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.302867 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.302871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.303004 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.303028 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.303102 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.303190 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":4}
I20260809 06:26:07.303547 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.297176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:37669 (local address 127.8.131.1:35815)
0809 06:26:07.297412 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.297420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.297635 (+ 215us) server_negotiation.cc:407] Connection header received
0809 06:26:07.297861 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.297869 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.297986 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.298106 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:07.298583 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.299589 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.301790 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.302714 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.302858 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.302957 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.303154 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.303295 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 06:26:07.304450 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.297787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:60875 (local address 127.8.131.2:34645)
0809 06:26:07.297994 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.298002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.298546 (+ 544us) server_negotiation.cc:407] Connection header received
0809 06:26:07.298684 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.298691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.298770 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.298899 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:07.299401 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.300323 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.302371 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.303552 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.303708 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.303827 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.304179 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.304272 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:26:07.304616 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.297642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:34645 (local address 127.8.131.3:60875)
0809 06:26:07.298412 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.298430 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.298560 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.298920 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.298923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.298944 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:07.299245 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.299251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.300454 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.300464 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.302223 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.302233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.303857 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.303864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.304014 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.304045 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.304147 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.304278 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":423,"threads_started":1}
I20260809 06:26:07.304570 20899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1011769765a4ae0aac0fd453b073fb8" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b047aac8e3714d96a76439e4139b6726" is_pre_election: true
I20260809 06:26:07.305235 20899 raft_consensus.cc:2454] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 0.
I20260809 06:26:07.305670 21097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1011769765a4ae0aac0fd453b073fb8" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" is_pre_election: true
I20260809 06:26:07.305856 21204 leader_election.cc:302] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 194b90f12683478ba4a0e39cc1bc6d96, b047aac8e3714d96a76439e4139b6726; no voters:
I20260809 06:26:07.306239 21424 raft_consensus.cc:2781] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:26:07.306200 21097 raft_consensus.cc:2454] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 0.
I20260809 06:26:07.306391 21424 raft_consensus.cc:482] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:26:07.306520 21424 raft_consensus.cc:3037] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:07.308837 21424 raft_consensus.cc:504] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.309374 21424 leader_election.cc:288] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 election: Requested vote from peers b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815), 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645)
I20260809 06:26:07.309805 20899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1011769765a4ae0aac0fd453b073fb8" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b047aac8e3714d96a76439e4139b6726"
I20260809 06:26:07.309867 21097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1011769765a4ae0aac0fd453b073fb8" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e"
I20260809 06:26:07.310001 20899 raft_consensus.cc:3037] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:07.310103 21097 raft_consensus.cc:3037] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:07.312335 20899 raft_consensus.cc:2454] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 1.
I20260809 06:26:07.312359 21097 raft_consensus.cc:2454] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 1.
I20260809 06:26:07.312731 21204 leader_election.cc:302] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 194b90f12683478ba4a0e39cc1bc6d96, b047aac8e3714d96a76439e4139b6726; no voters:
I20260809 06:26:07.313012 21424 raft_consensus.cc:2781] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:07.313462 21424 raft_consensus.cc:686] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 LEADER]: Becoming Leader. State: Replica: 194b90f12683478ba4a0e39cc1bc6d96, State: Running, Role: LEADER
I20260809 06:26:07.314008 21424 consensus_queue.cc:237] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.318709 20664 catalog_manager.cc:5495] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3). New cstate: current_term: 1 leader_uuid: "194b90f12683478ba4a0e39cc1bc6d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } health_report { overall_health: HEALTHY } } }
W20260809 06:26:07.342298 20970 tablet.cc:2369] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:26:07.352676 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.347324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35104 (local address 127.8.131.3:33689)
0809 06:26:07.347520 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.347526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.347628 (+ 102us) server_negotiation.cc:407] Connection header received
0809 06:26:07.347947 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.347952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.348039 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.348194 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:07.348538 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.349238 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.350550 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.350962 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.351098 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.351521 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:07.352106 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:07.352270 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.352342 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.352423 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:26:07.360265 21097 raft_consensus.cc:1261] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Refusing update from remote peer 194b90f12683478ba4a0e39cc1bc6d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:07.360302 20899 raft_consensus.cc:1261] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Refusing update from remote peer 194b90f12683478ba4a0e39cc1bc6d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:07.361196 21426 consensus_queue.cc:1035] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:07.361546 21424 consensus_queue.cc:1035] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:07.373716 21432 mvcc.cc:205] Tried to move back new op lower bound from 7316507719098437632 to 7316507718921777152. Current Snapshot: MvccSnapshot[applied={T|T < 7316507719098437632}]
I20260809 06:26:07.375087 21435 mvcc.cc:205] Tried to move back new op lower bound from 7316507719098437632 to 7316507718921777152. Current Snapshot: MvccSnapshot[applied={T|T < 7316507719098437632}]
I20260809 06:26:07.391469 20665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:07.391801 20665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:07.396023 20665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564: Sending DeleteTablet for 3 replicas of tablet b1011769765a4ae0aac0fd453b073fb8
I20260809 06:26:07.397239 21071 tablet_service.cc:1514] Processing DeleteTablet for tablet b1011769765a4ae0aac0fd453b073fb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:07 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:56458
I20260809 06:26:07.397324 20873 tablet_service.cc:1514] Processing DeleteTablet for tablet b1011769765a4ae0aac0fd453b073fb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:07 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:45150
I20260809 06:26:07.398649 21279 tablet_service.cc:1514] Processing DeleteTablet for tablet b1011769765a4ae0aac0fd453b073fb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:07 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:35088
I20260809 06:26:07.401062 20665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:26:07.401897 20665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:26:07.403898 21450 tablet_replica.cc:330] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: stopping tablet replica
I20260809 06:26:07.404434 21450 raft_consensus.cc:2229] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:07.405016 21450 raft_consensus.cc:2258] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:07.405985 21452 tablet_replica.cc:330] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: stopping tablet replica
I20260809 06:26:07.406373 21452 raft_consensus.cc:2229] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:26:07.407126 21452 raft_consensus.cc:2258] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:07.408138 21449 tablet_replica.cc:330] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: stopping tablet replica
I20260809 06:26:07.408519 21449 raft_consensus.cc:2229] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:07.408969 21449 raft_consensus.cc:2258] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:07.410056 21450 ts_tablet_manager.cc:1905] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:07.410892 21449 ts_tablet_manager.cc:1905] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:07.411733 21279 tablet_service.cc:1467] Processing CreateTablet for tablet d6c9e4f30c5e422d82255eac73d20f81 (DEFAULT_TABLE table=test-table [id=2e49d8ded7c6417a92a5bbc4d8d5146d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:07.412201 21279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c9e4f30c5e422d82255eac73d20f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:07.416078 20873 tablet_service.cc:1467] Processing CreateTablet for tablet d6c9e4f30c5e422d82255eac73d20f81 (DEFAULT_TABLE table=test-table [id=2e49d8ded7c6417a92a5bbc4d8d5146d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:07.416540 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c9e4f30c5e422d82255eac73d20f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:07.416998 21450 ts_tablet_manager.cc:1918] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:07.417145 21450 log.cc:1196] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/b1011769765a4ae0aac0fd453b073fb8
I20260809 06:26:07.417621 21449 ts_tablet_manager.cc:1918] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:07.417721 21450 ts_tablet_manager.cc:1939] T b1011769765a4ae0aac0fd453b073fb8 P b047aac8e3714d96a76439e4139b6726: Deleting consensus metadata
I20260809 06:26:07.417778 21449 log.cc:1196] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/b1011769765a4ae0aac0fd453b073fb8
I20260809 06:26:07.418339 21449 ts_tablet_manager.cc:1939] T b1011769765a4ae0aac0fd453b073fb8 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting consensus metadata
I20260809 06:26:07.419267 21452 ts_tablet_manager.cc:1905] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:07.420439 20640 catalog_manager.cc:4841] TS 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645): tablet b1011769765a4ae0aac0fd453b073fb8 (table test-table [id=44e5602f56f74cb2a692dea253853166]) successfully deleted
I20260809 06:26:07.421795 20640 catalog_manager.cc:4841] TS b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815): tablet b1011769765a4ae0aac0fd453b073fb8 (table test-table [id=44e5602f56f74cb2a692dea253853166]) successfully deleted
I20260809 06:26:07.423394 21071 tablet_service.cc:1467] Processing CreateTablet for tablet d6c9e4f30c5e422d82255eac73d20f81 (DEFAULT_TABLE table=test-table [id=2e49d8ded7c6417a92a5bbc4d8d5146d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:07.423878 21071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c9e4f30c5e422d82255eac73d20f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:07.427312 21452 ts_tablet_manager.cc:1918] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:07.427487 21452 log.cc:1196] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/b1011769765a4ae0aac0fd453b073fb8
I20260809 06:26:07.427724 21417 tablet_bootstrap.cc:492] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: Bootstrap starting.
I20260809 06:26:07.428084 21452 ts_tablet_manager.cc:1939] T b1011769765a4ae0aac0fd453b073fb8 P 194b90f12683478ba4a0e39cc1bc6d96: Deleting consensus metadata
I20260809 06:26:07.429391 21417 tablet_bootstrap.cc:654] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:07.430357 21414 tablet_bootstrap.cc:492] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Bootstrap starting.
I20260809 06:26:07.430827 21418 tablet_bootstrap.cc:492] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: Bootstrap starting.
I20260809 06:26:07.431932 21414 tablet_bootstrap.cc:654] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:07.432515 21418 tablet_bootstrap.cc:654] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:07.433741 21414 tablet_bootstrap.cc:492] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: No bootstrap required, opened a new log
I20260809 06:26:07.433837 21414 ts_tablet_manager.cc:1397] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 06:26:07.434391 21414 raft_consensus.cc:348] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.434538 21414 raft_consensus.cc:374] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:07.434597 21414 raft_consensus.cc:729] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8226868cd3dc40f19e8b3cb9cf5d4a3e, State: Initialized, Role: FOLLOWER
I20260809 06:26:07.434872 21414 consensus_queue.cc:260] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.435465 21414 ts_tablet_manager.cc:1428] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 06:26:07.435794 20644 catalog_manager.cc:4841] TS 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3:33689): tablet b1011769765a4ae0aac0fd453b073fb8 (table test-table [id=44e5602f56f74cb2a692dea253853166]) successfully deleted
I20260809 06:26:07.436950 21417 tablet_bootstrap.cc:492] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: No bootstrap required, opened a new log
I20260809 06:26:07.437198 21417 ts_tablet_manager.cc:1397] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260809 06:26:07.438028 21417 raft_consensus.cc:348] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.438827 21417 raft_consensus.cc:374] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:07.438949 21417 raft_consensus.cc:729] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b047aac8e3714d96a76439e4139b6726, State: Initialized, Role: FOLLOWER
I20260809 06:26:07.439225 21417 consensus_queue.cc:260] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.440794 21417 ts_tablet_manager.cc:1428] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 06:26:07.448984 21418 tablet_bootstrap.cc:492] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: No bootstrap required, opened a new log
I20260809 06:26:07.449196 21418 ts_tablet_manager.cc:1397] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.009s
I20260809 06:26:07.450096 21418 raft_consensus.cc:348] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.450335 21418 raft_consensus.cc:374] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:07.450441 21418 raft_consensus.cc:729] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194b90f12683478ba4a0e39cc1bc6d96, State: Initialized, Role: FOLLOWER
I20260809 06:26:07.450701 21418 consensus_queue.cc:260] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.451243 21418 ts_tablet_manager.cc:1428] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 06:26:07.617400 21424 raft_consensus.cc:482] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:26:07.617656 21424 raft_consensus.cc:504] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.618444 21424 leader_election.cc:288] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815), 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645)
I20260809 06:26:07.619141 21097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c9e4f30c5e422d82255eac73d20f81" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" is_pre_election: true
I20260809 06:26:07.619169 20899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c9e4f30c5e422d82255eac73d20f81" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b047aac8e3714d96a76439e4139b6726" is_pre_election: true
I20260809 06:26:07.619436 21097 raft_consensus.cc:2454] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 0.
I20260809 06:26:07.619475 20899 raft_consensus.cc:2454] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 0.
I20260809 06:26:07.619925 21204 leader_election.cc:302] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 194b90f12683478ba4a0e39cc1bc6d96, 8226868cd3dc40f19e8b3cb9cf5d4a3e; no voters:
I20260809 06:26:07.620229 21424 raft_consensus.cc:2781] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:26:07.620361 21424 raft_consensus.cc:482] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:26:07.620479 21424 raft_consensus.cc:3037] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:07.623384 21424 raft_consensus.cc:504] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.624061 21424 leader_election.cc:288] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 election: Requested vote from peers b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815), 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645)
I20260809 06:26:07.624521 20899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c9e4f30c5e422d82255eac73d20f81" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b047aac8e3714d96a76439e4139b6726"
I20260809 06:26:07.624751 20899 raft_consensus.cc:3037] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:07.624680 21097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c9e4f30c5e422d82255eac73d20f81" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e"
I20260809 06:26:07.624900 21097 raft_consensus.cc:3037] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:07.626839 20899 raft_consensus.cc:2454] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 1.
I20260809 06:26:07.627287 21204 leader_election.cc:302] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 194b90f12683478ba4a0e39cc1bc6d96, b047aac8e3714d96a76439e4139b6726; no voters:
I20260809 06:26:07.627593 21424 raft_consensus.cc:2781] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:07.627833 21424 raft_consensus.cc:686] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 LEADER]: Becoming Leader. State: Replica: 194b90f12683478ba4a0e39cc1bc6d96, State: Running, Role: LEADER
I20260809 06:26:07.628024 21097 raft_consensus.cc:2454] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 1.
I20260809 06:26:07.628166 21424 consensus_queue.cc:237] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.631152 20663 catalog_manager.cc:5495] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3). New cstate: current_term: 1 leader_uuid: "194b90f12683478ba4a0e39cc1bc6d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } health_report { overall_health: HEALTHY } } }
Aug 09 06:26:07 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256765, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.62@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:26:07.716045 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.694272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39546 (local address 127.8.131.62:40179)
0809 06:26:07.696663 (+ 2391us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.696667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.696687 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:26:07.696730 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.696733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.696783 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.696870 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:07.698463 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.699247 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.701182 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.701600 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.701695 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.708781 (+ 7086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:07.708786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:07.708790 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:07.708821 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:07.712800 (+ 3979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:07.713812 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:07.713816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:07.713818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:07.713943 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:07.714402 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:07.714405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:07.714407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:07.714679 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:07.714804 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.715343 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.715774 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":171,"server-negotiator.queue_time_us":2294,"thread_start_us":2102,"threads_started":1}
I20260809 06:26:07.716135 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.694212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40179 (local address 127.0.0.1:39546)
0809 06:26:07.696220 (+ 2008us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.696243 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.696506 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.696968 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.696977 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.697981 (+ 1004us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:07.698289 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.698303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.699406 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.699411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.701024 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.701043 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.701792 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.701800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.701989 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.705422 (+ 3433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:07.705451 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:07.708560 (+ 3109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:07.712955 (+ 4395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:07.712970 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:07.712979 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:07.713671 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:07.714066 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:07.714071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:07.714075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:07.714265 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:07.714829 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:07.714837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:07.715050 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.715252 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.715585 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1814,"thread_start_us":1638,"threads_started":1}
I20260809 06:26:07.722061 20663 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:39546:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 06:26:07.723397 20663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:26:07.730895 20873 tablet_service.cc:1467] Processing CreateTablet for tablet 072fef4c3e88429ca205f30cbe6cc03a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b206229830046f5898fcda1e816cd2e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 06:26:07.731549 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072fef4c3e88429ca205f30cbe6cc03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:07.731350 21279 tablet_service.cc:1467] Processing CreateTablet for tablet 072fef4c3e88429ca205f30cbe6cc03a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b206229830046f5898fcda1e816cd2e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 06:26:07.732018 21279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072fef4c3e88429ca205f30cbe6cc03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:07.736282 21071 tablet_service.cc:1467] Processing CreateTablet for tablet 072fef4c3e88429ca205f30cbe6cc03a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b206229830046f5898fcda1e816cd2e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 06:26:07.736920 21071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072fef4c3e88429ca205f30cbe6cc03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:07.739117 21417 tablet_bootstrap.cc:492] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: Bootstrap starting.
I20260809 06:26:07.739439 21418 tablet_bootstrap.cc:492] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: Bootstrap starting.
I20260809 06:26:07.740931 21417 tablet_bootstrap.cc:654] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:07.741340 21418 tablet_bootstrap.cc:654] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:07.746687 21414 tablet_bootstrap.cc:492] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Bootstrap starting.
I20260809 06:26:07.748487 21414 tablet_bootstrap.cc:654] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:07.750908 21418 tablet_bootstrap.cc:492] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: No bootstrap required, opened a new log
I20260809 06:26:07.751124 21418 ts_tablet_manager.cc:1397] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260809 06:26:07.751514 21414 tablet_bootstrap.cc:492] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: No bootstrap required, opened a new log
I20260809 06:26:07.751752 21414 ts_tablet_manager.cc:1397] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 06:26:07.752226 21418 raft_consensus.cc:348] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.752547 21418 raft_consensus.cc:374] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:07.752643 21418 raft_consensus.cc:729] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194b90f12683478ba4a0e39cc1bc6d96, State: Initialized, Role: FOLLOWER
I20260809 06:26:07.752766 21414 raft_consensus.cc:348] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.753084 21414 raft_consensus.cc:374] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:07.752990 21418 consensus_queue.cc:260] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.753479 21424 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.753824 21417 tablet_bootstrap.cc:492] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: No bootstrap required, opened a new log
I20260809 06:26:07.753842 21424 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:26:07.753995 21417 ts_tablet_manager.cc:1397] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260809 06:26:07.754354 21418 ts_tablet_manager.cc:1428] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 06:26:07.754740 21417 raft_consensus.cc:348] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.754973 21417 raft_consensus.cc:374] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:07.755080 21417 raft_consensus.cc:729] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b047aac8e3714d96a76439e4139b6726, State: Initialized, Role: FOLLOWER
I20260809 06:26:07.755374 21417 consensus_queue.cc:260] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.753295 21414 raft_consensus.cc:729] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8226868cd3dc40f19e8b3cb9cf5d4a3e, State: Initialized, Role: FOLLOWER
I20260809 06:26:07.755882 21422 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.756232 21422 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:26:07.756699 21414 consensus_queue.cc:260] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.757513 21417 ts_tablet_manager.cc:1428] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 06:26:07.758179 21423 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.758518 21423 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:26:07.759621 21414 ts_tablet_manager.cc:1428] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260809 06:26:07.780975 21422 raft_consensus.cc:482] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:26:07.781205 21422 raft_consensus.cc:504] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.782855 21422 leader_election.cc:288] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645), 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3:33689)
I20260809 06:26:07.792059 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.784025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:51521 (local address 127.8.131.2:34645)
0809 06:26:07.784222 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.784228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.785452 (+ 1224us) server_negotiation.cc:407] Connection header received
0809 06:26:07.785718 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.785724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.785815 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.785879 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:07.786884 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.787983 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.790006 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.790997 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.791119 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.791220 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.791572 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.791742 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:26:07.792913 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.783394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:45799 (local address 127.8.131.3:33689)
0809 06:26:07.783825 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.783834 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.785192 (+ 1358us) server_negotiation.cc:407] Connection header received
0809 06:26:07.785591 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.785597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.785686 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.785757 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:07.786623 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.787900 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.790544 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.791733 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.791882 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.791974 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.792475 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.792615 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260809 06:26:07.792960 21472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.783200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:34645 (local address 127.8.131.1:51521)
0809 06:26:07.785352 (+ 2152us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.785370 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.785556 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.786089 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.786098 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.786143 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:07.786679 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.786694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.788112 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.788117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.789848 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.789861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.791217 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.791223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.791394 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.791428 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.791526 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.791680 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2025,"mutex_wait_us":122,"thread_start_us":1453,"threads_started":1}
I20260809 06:26:07.793748 21097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072fef4c3e88429ca205f30cbe6cc03a" candidate_uuid: "b047aac8e3714d96a76439e4139b6726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" is_pre_election: true
I20260809 06:26:07.793879 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.783179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:33689 (local address 127.8.131.1:45799)
0809 06:26:07.785073 (+ 1894us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.785097 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.785346 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.785989 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.785992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.786012 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:07.786436 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.786447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.788059 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.788066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.790368 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.790383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.792090 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.792096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.792296 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.792329 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.793225 (+ 896us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.793508 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1694,"mutex_wait_us":114,"spinlock_wait_cycles":4992,"thread_start_us":1853,"threads_started":1}
I20260809 06:26:07.794030 21097 raft_consensus.cc:2454] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b047aac8e3714d96a76439e4139b6726 in term 0.
I20260809 06:26:07.794533 20794 leader_election.cc:302] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8226868cd3dc40f19e8b3cb9cf5d4a3e, b047aac8e3714d96a76439e4139b6726; no voters:
I20260809 06:26:07.794574 21310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072fef4c3e88429ca205f30cbe6cc03a" candidate_uuid: "b047aac8e3714d96a76439e4139b6726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "194b90f12683478ba4a0e39cc1bc6d96" is_pre_election: true
I20260809 06:26:07.794922 21422 raft_consensus.cc:2781] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:26:07.795024 21422 raft_consensus.cc:482] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:26:07.795086 21422 raft_consensus.cc:3037] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:07.795003 21310 raft_consensus.cc:2454] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b047aac8e3714d96a76439e4139b6726 in term 0.
I20260809 06:26:07.797529 21422 raft_consensus.cc:504] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.798172 21422 leader_election.cc:288] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [CANDIDATE]: Term 1 election: Requested vote from peers 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645), 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3:33689)
I20260809 06:26:07.798589 21097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072fef4c3e88429ca205f30cbe6cc03a" candidate_uuid: "b047aac8e3714d96a76439e4139b6726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e"
I20260809 06:26:07.798727 21310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072fef4c3e88429ca205f30cbe6cc03a" candidate_uuid: "b047aac8e3714d96a76439e4139b6726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "194b90f12683478ba4a0e39cc1bc6d96"
I20260809 06:26:07.798835 21097 raft_consensus.cc:3037] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:07.798960 21310 raft_consensus.cc:3037] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:07.801898 21097 raft_consensus.cc:2454] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b047aac8e3714d96a76439e4139b6726 in term 1.
I20260809 06:26:07.802301 20794 leader_election.cc:302] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8226868cd3dc40f19e8b3cb9cf5d4a3e, b047aac8e3714d96a76439e4139b6726; no voters:
I20260809 06:26:07.802345 21310 raft_consensus.cc:2454] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b047aac8e3714d96a76439e4139b6726 in term 1.
I20260809 06:26:07.802603 21422 raft_consensus.cc:2781] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:07.803128 21422 raft_consensus.cc:686] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [term 1 LEADER]: Becoming Leader. State: Replica: b047aac8e3714d96a76439e4139b6726, State: Running, Role: LEADER
I20260809 06:26:07.803603 21422 consensus_queue.cc:237] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:07.806258 21475 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: TxnStatusTablet state changed. Reason: New leader b047aac8e3714d96a76439e4139b6726. Latest consensus state: current_term: 1 leader_uuid: "b047aac8e3714d96a76439e4139b6726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.806607 21475 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: This TxnStatusTablet replica's current role is: LEADER
I20260809 06:26:07.807255 21477 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 06:26:07.807504 21477 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 06:26:07.807751 20663 catalog_manager.cc:5495] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 reported cstate change: term changed from 0 to 1, leader changed from <none> to b047aac8e3714d96a76439e4139b6726 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "b047aac8e3714d96a76439e4139b6726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } health_report { overall_health: UNKNOWN } } }
I20260809 06:26:07.831485 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.821003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35815 (local address 127.0.0.1:45152)
0809 06:26:07.821526 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.821546 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.821673 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.824643 (+ 2970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.824648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.824664 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:26:07.824890 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.824903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.826554 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.826560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.828257 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.828300 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.829399 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.829408 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.829645 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.829721 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:26:07.830849 (+ 1128us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:26:07.830873 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.830971 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.831149 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321}
I20260809 06:26:07.831507 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.821551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45152 (local address 127.8.131.1:35815)
0809 06:26:07.824151 (+ 2600us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.824157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.824183 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:26:07.824280 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.824285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.824394 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.824584 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:07.825084 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.826373 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.828427 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.829209 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.829344 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.829975 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:07.830715 (+ 740us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:07.830852 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.831007 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.831090 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2444,"thread_start_us":2268,"threads_started":1}
I20260809 06:26:07.836433 21097 raft_consensus.cc:1261] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Refusing update from remote peer b047aac8e3714d96a76439e4139b6726: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:07.836521 21310 raft_consensus.cc:1261] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Refusing update from remote peer b047aac8e3714d96a76439e4139b6726: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:07.837208 21422 consensus_queue.cc:1035] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:07.837507 21475 consensus_queue.cc:1035] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726 [LEADER]: Connected to new peer: Peer: permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:07.851385 21424 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: TxnStatusTablet state changed. Reason: New leader b047aac8e3714d96a76439e4139b6726. Latest consensus state: current_term: 1 leader_uuid: "b047aac8e3714d96a76439e4139b6726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.854254 21424 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:26:07.856060 21423 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: TxnStatusTablet state changed. Reason: New leader b047aac8e3714d96a76439e4139b6726. Latest consensus state: current_term: 1 leader_uuid: "b047aac8e3714d96a76439e4139b6726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.856389 21423 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:26:07.857658 21422 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b047aac8e3714d96a76439e4139b6726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.857954 21422 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: This TxnStatusTablet replica's current role is: LEADER
I20260809 06:26:07.858417 21424 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b047aac8e3714d96a76439e4139b6726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.858750 21424 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P 194b90f12683478ba4a0e39cc1bc6d96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:26:07.861829 21475 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b047aac8e3714d96a76439e4139b6726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.862164 21475 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P b047aac8e3714d96a76439e4139b6726: This TxnStatusTablet replica's current role is: LEADER
I20260809 06:26:07.863178 21423 tablet_replica.cc:439] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b047aac8e3714d96a76439e4139b6726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } } }
I20260809 06:26:07.863415 21423 tablet_replica.cc:442] T 072fef4c3e88429ca205f30cbe6cc03a P 8226868cd3dc40f19e8b3cb9cf5d4a3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 06:26:07.887935 21493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.881360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35815 (local address 127.8.131.3:52579)
0809 06:26:07.882005 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.882025 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.882208 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.882765 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.882770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.882782 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:26:07.883050 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.883060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.884666 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.884670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.885748 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.885759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.886753 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.886757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.886885 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.886943 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:26:07.887374 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:26:07.887383 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.887445 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.887551 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":353,"threads_started":1}
I20260809 06:26:07.887930 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.881389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:52579 (local address 127.8.131.1:35815)
0809 06:26:07.881580 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.881586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.882104 (+ 518us) server_negotiation.cc:407] Connection header received
0809 06:26:07.882394 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.882405 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.882496 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.882557 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:07.883237 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.884522 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.885900 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.886580 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.886714 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.887131 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:07.887260 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:07.887367 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.887499 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.887587 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 06:26:07.898245 21097 raft_consensus.cc:1261] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Refusing update from remote peer 194b90f12683478ba4a0e39cc1bc6d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:07.898412 20899 raft_consensus.cc:1261] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Refusing update from remote peer 194b90f12683478ba4a0e39cc1bc6d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:07.898850 21438 consensus_queue.cc:1035] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:07.899185 21424 consensus_queue.cc:1035] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:07.907253 21430 mvcc.cc:205] Tried to move back new op lower bound from 7316507721305673728 to 7316507720207937536. Current Snapshot: MvccSnapshot[applied={T|T < 7316507721305673728}]
I20260809 06:26:07.956023 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.945182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:42319 (local address 127.8.131.3:33689)
0809 06:26:07.945334 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:07.945339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:07.947993 (+ 2654us) server_negotiation.cc:407] Connection header received
0809 06:26:07.948267 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:07.948273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:07.948371 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:07.948429 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:07.949174 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.950214 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.953675 (+ 3461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.954357 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.954485 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.955140 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:07.955301 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:07.955384 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:07.955621 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 06:26:07.955735 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 06:26:07.956514 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:07.945255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:33689 (local address 127.8.131.1:42319)
0809 06:26:07.947891 (+ 2636us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:07.947913 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:07.948098 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:07.948617 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:07.948622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:07.948634 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:26:07.948978 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.948991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.950409 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.950414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.953504 (+ 3090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:07.953521 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:07.954559 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:07.954566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:07.954791 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:07.954895 (+ 104us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:26:07.955476 (+ 581us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:26:07.955487 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 06:26:07.955573 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 06:26:07.955752 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2431,"thread_start_us":2235,"threads_started":1}
I20260809 06:26:07.991829 20663 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:07.992066 20663 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:07.995271 20663 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564: Sending DeleteTablet for 3 replicas of tablet d6c9e4f30c5e422d82255eac73d20f81
I20260809 06:26:07.996169 20873 tablet_service.cc:1514] Processing DeleteTablet for tablet d6c9e4f30c5e422d82255eac73d20f81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:07 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:45150
I20260809 06:26:07.996295 21071 tablet_service.cc:1514] Processing DeleteTablet for tablet d6c9e4f30c5e422d82255eac73d20f81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:07 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:56458
I20260809 06:26:07.996402 21279 tablet_service.cc:1514] Processing DeleteTablet for tablet d6c9e4f30c5e422d82255eac73d20f81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:07 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:35088
I20260809 06:26:08.005998 21508 tablet_replica.cc:330] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: stopping tablet replica
I20260809 06:26:08.006748 21503 tablet_replica.cc:330] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: stopping tablet replica
I20260809 06:26:08.007210 21508 raft_consensus.cc:2229] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:26:08.007597 21503 raft_consensus.cc:2229] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:08.008215 21508 raft_consensus.cc:2258] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:08.008677 21503 raft_consensus.cc:2258] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:08.010387 21508 ts_tablet_manager.cc:1905] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:08.011602 21507 tablet_replica.cc:330] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: stopping tablet replica
I20260809 06:26:08.011852 21507 raft_consensus.cc:2229] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:08.012087 21507 raft_consensus.cc:2258] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:08.013258 21507 ts_tablet_manager.cc:1905] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:08.016147 21503 ts_tablet_manager.cc:1905] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:08.018347 21508 ts_tablet_manager.cc:1918] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:26:08.018536 21508 log.cc:1196] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/d6c9e4f30c5e422d82255eac73d20f81
I20260809 06:26:08.019150 21508 ts_tablet_manager.cc:1939] T d6c9e4f30c5e422d82255eac73d20f81 P 194b90f12683478ba4a0e39cc1bc6d96: Deleting consensus metadata
I20260809 06:26:08.020318 21507 ts_tablet_manager.cc:1918] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:26:08.020467 21507 log.cc:1196] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/d6c9e4f30c5e422d82255eac73d20f81
I20260809 06:26:08.020526 20644 catalog_manager.cc:4841] TS 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3:33689): tablet d6c9e4f30c5e422d82255eac73d20f81 (table test-table [id=2e49d8ded7c6417a92a5bbc4d8d5146d]) successfully deleted
I20260809 06:26:08.021000 21507 ts_tablet_manager.cc:1939] T d6c9e4f30c5e422d82255eac73d20f81 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting consensus metadata
I20260809 06:26:08.023360 20640 catalog_manager.cc:4841] TS 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645): tablet d6c9e4f30c5e422d82255eac73d20f81 (table test-table [id=2e49d8ded7c6417a92a5bbc4d8d5146d]) successfully deleted
I20260809 06:26:08.024083 21503 ts_tablet_manager.cc:1918] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:26:08.024237 21503 log.cc:1196] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/d6c9e4f30c5e422d82255eac73d20f81
I20260809 06:26:08.024721 21503 ts_tablet_manager.cc:1939] T d6c9e4f30c5e422d82255eac73d20f81 P b047aac8e3714d96a76439e4139b6726: Deleting consensus metadata
I20260809 06:26:08.025810 20640 catalog_manager.cc:4841] TS b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815): tablet d6c9e4f30c5e422d82255eac73d20f81 (table test-table [id=2e49d8ded7c6417a92a5bbc4d8d5146d]) successfully deleted
Aug 09 06:26:08 dist-test-slave-mjjr krb5kdc[20601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 06:26:08 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:08 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 06:26:08.073714 21466 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:08.061706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.8.131.62:40179)
0809 06:26:08.062018 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.062025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.062445 (+ 420us) server_negotiation.cc:407] Connection header received
0809 06:26:08.062700 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.062707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.062802 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.062903 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:08.064684 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.065434 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.067270 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.067818 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.067913 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.073448 (+ 5535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39560: BlockingRecv error: recv got EOF from 127.0.0.1:39560 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 06:26:08 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM
I20260809 06:26:08.092116 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.076653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.8.131.62:40179)
0809 06:26:08.076846 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.076853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.077182 (+ 329us) server_negotiation.cc:407] Connection header received
0809 06:26:08.077320 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.077325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.077407 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.077520 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:08.078715 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.079896 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.081249 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.081936 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.082058 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.085912 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:08.085918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:08.085923 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:08.085965 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:08.089344 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:08.090010 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:08.090016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:08.090019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:08.090198 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:08.090559 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:08.090564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:08.090568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:08.090932 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:08.091201 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:08.091444 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 06:26:08.091708 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 06:26:08.096246 20663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:26:08.097043 20663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:26:08.104075 20873 tablet_service.cc:1467] Processing CreateTablet for tablet faa8b8fd92f24e869e99a56dad7877f5 (DEFAULT_TABLE table=test-table [id=a6b6f2de222a4c7b932d6a1da9a8f3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:08.104242 21279 tablet_service.cc:1467] Processing CreateTablet for tablet faa8b8fd92f24e869e99a56dad7877f5 (DEFAULT_TABLE table=test-table [id=a6b6f2de222a4c7b932d6a1da9a8f3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:08.104619 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faa8b8fd92f24e869e99a56dad7877f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:08.104673 21279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faa8b8fd92f24e869e99a56dad7877f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:08.108379 21071 tablet_service.cc:1467] Processing CreateTablet for tablet faa8b8fd92f24e869e99a56dad7877f5 (DEFAULT_TABLE table=test-table [id=a6b6f2de222a4c7b932d6a1da9a8f3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:08.108841 21071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faa8b8fd92f24e869e99a56dad7877f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:08.111663 21417 tablet_bootstrap.cc:492] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: Bootstrap starting.
I20260809 06:26:08.113545 21417 tablet_bootstrap.cc:654] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:08.114449 21418 tablet_bootstrap.cc:492] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: Bootstrap starting.
I20260809 06:26:08.115926 21414 tablet_bootstrap.cc:492] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Bootstrap starting.
I20260809 06:26:08.116158 21418 tablet_bootstrap.cc:654] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:08.117882 21414 tablet_bootstrap.cc:654] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:08.124141 21414 tablet_bootstrap.cc:492] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: No bootstrap required, opened a new log
I20260809 06:26:08.124339 21414 ts_tablet_manager.cc:1397] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260809 06:26:08.124982 21414 raft_consensus.cc:348] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.125209 21414 raft_consensus.cc:374] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:08.125315 21414 raft_consensus.cc:729] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8226868cd3dc40f19e8b3cb9cf5d4a3e, State: Initialized, Role: FOLLOWER
I20260809 06:26:08.125545 21414 consensus_queue.cc:260] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.126080 21414 ts_tablet_manager.cc:1428] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 06:26:08.127701 21417 tablet_bootstrap.cc:492] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: No bootstrap required, opened a new log
I20260809 06:26:08.127945 21417 ts_tablet_manager.cc:1397] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.003s
I20260809 06:26:08.128926 21417 raft_consensus.cc:348] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.129302 21417 raft_consensus.cc:374] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:08.129479 21417 raft_consensus.cc:729] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b047aac8e3714d96a76439e4139b6726, State: Initialized, Role: FOLLOWER
I20260809 06:26:08.129559 21418 tablet_bootstrap.cc:492] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: No bootstrap required, opened a new log
I20260809 06:26:08.129735 21418 ts_tablet_manager.cc:1397] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.005s
I20260809 06:26:08.129951 21417 consensus_queue.cc:260] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.130564 21418 raft_consensus.cc:348] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.130843 21418 raft_consensus.cc:374] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:08.130937 21418 raft_consensus.cc:729] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194b90f12683478ba4a0e39cc1bc6d96, State: Initialized, Role: FOLLOWER
I20260809 06:26:08.131204 21418 consensus_queue.cc:260] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.131366 21417 ts_tablet_manager.cc:1428] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 06:26:08.131935 21418 ts_tablet_manager.cc:1428] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 06:26:08.195976 21496 raft_consensus.cc:482] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:26:08.196282 21496 raft_consensus.cc:504] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.197307 21496 leader_election.cc:288] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815), 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645)
I20260809 06:26:08.198143 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa8b8fd92f24e869e99a56dad7877f5" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" is_pre_election: true
I20260809 06:26:08.198523 21096 raft_consensus.cc:2454] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 0.
I20260809 06:26:08.198496 20899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa8b8fd92f24e869e99a56dad7877f5" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b047aac8e3714d96a76439e4139b6726" is_pre_election: true
I20260809 06:26:08.198812 20899 raft_consensus.cc:2454] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 0.
I20260809 06:26:08.199136 21204 leader_election.cc:302] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 194b90f12683478ba4a0e39cc1bc6d96, 8226868cd3dc40f19e8b3cb9cf5d4a3e; no voters:
I20260809 06:26:08.199605 21496 raft_consensus.cc:2781] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:26:08.199813 21496 raft_consensus.cc:482] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:26:08.199949 21496 raft_consensus.cc:3037] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:08.202664 21496 raft_consensus.cc:504] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.203358 21496 leader_election.cc:288] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 election: Requested vote from peers b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815), 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645)
I20260809 06:26:08.204041 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa8b8fd92f24e869e99a56dad7877f5" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e"
I20260809 06:26:08.204293 21096 raft_consensus.cc:3037] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:08.204392 20899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa8b8fd92f24e869e99a56dad7877f5" candidate_uuid: "194b90f12683478ba4a0e39cc1bc6d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b047aac8e3714d96a76439e4139b6726"
I20260809 06:26:08.204656 20899 raft_consensus.cc:3037] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:08.208374 20899 raft_consensus.cc:2454] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 1.
I20260809 06:26:08.208029 21096 raft_consensus.cc:2454] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 194b90f12683478ba4a0e39cc1bc6d96 in term 1.
I20260809 06:26:08.209003 21204 leader_election.cc:302] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 194b90f12683478ba4a0e39cc1bc6d96, b047aac8e3714d96a76439e4139b6726; no voters:
I20260809 06:26:08.210207 21496 raft_consensus.cc:2781] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:08.210489 21496 raft_consensus.cc:686] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 LEADER]: Becoming Leader. State: Replica: 194b90f12683478ba4a0e39cc1bc6d96, State: Running, Role: LEADER
I20260809 06:26:08.210881 21496 consensus_queue.cc:237] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.214187 20663 catalog_manager.cc:5495] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3). New cstate: current_term: 1 leader_uuid: "194b90f12683478ba4a0e39cc1bc6d96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } health_report { overall_health: HEALTHY } } }
I20260809 06:26:08.246143 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.239606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.8.131.3:33689)
0809 06:26:08.239918 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.239922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.239965 (+ 43us) server_negotiation.cc:407] Connection header received
0809 06:26:08.240229 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.240233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.240317 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.240385 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:08.240956 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.241900 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.244002 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.244526 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.244649 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.245228 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:08.245371 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:08.245457 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:08.245653 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 06:26:08.245734 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260809 06:26:08.249432 20899 raft_consensus.cc:1261] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Refusing update from remote peer 194b90f12683478ba4a0e39cc1bc6d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:08.249434 21096 raft_consensus.cc:1261] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Refusing update from remote peer 194b90f12683478ba4a0e39cc1bc6d96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:08.250095 21496 consensus_queue.cc:1035] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:08.250463 21438 consensus_queue.cc:1035] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:08.269133 20665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:08.269456 20665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:08.272773 20665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564: Sending DeleteTablet for 3 replicas of tablet faa8b8fd92f24e869e99a56dad7877f5
I20260809 06:26:08.273881 21279 tablet_service.cc:1514] Processing DeleteTablet for tablet faa8b8fd92f24e869e99a56dad7877f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:08 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:35088
I20260809 06:26:08.273962 20873 tablet_service.cc:1514] Processing DeleteTablet for tablet faa8b8fd92f24e869e99a56dad7877f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:08 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:45150
I20260809 06:26:08.274394 21071 tablet_service.cc:1514] Processing DeleteTablet for tablet faa8b8fd92f24e869e99a56dad7877f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:08 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:56458
I20260809 06:26:08.274820 21508 tablet_replica.cc:330] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: stopping tablet replica
I20260809 06:26:08.275084 21508 raft_consensus.cc:2229] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:26:08.275220 20665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 06:26:08.275471 21508 raft_consensus.cc:2258] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:08.275863 21503 tablet_replica.cc:330] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: stopping tablet replica
W20260809 06:26:08.276023 20665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:26:08.276129 21503 raft_consensus.cc:2229] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:08.276436 21503 raft_consensus.cc:2258] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:08.276811 21507 tablet_replica.cc:330] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: stopping tablet replica
I20260809 06:26:08.277061 21507 raft_consensus.cc:2229] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:08.277354 21507 raft_consensus.cc:2258] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:08.278645 21508 ts_tablet_manager.cc:1905] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:08.284644 21279 tablet_service.cc:1467] Processing CreateTablet for tablet c9e7048a049544ebbbcbd2c184a45e5f (DEFAULT_TABLE table=test-table [id=659f35b1ad6143ca88c1877bcd65ec6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:08.285066 21279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e7048a049544ebbbcbd2c184a45e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:08.285171 21071 tablet_service.cc:1467] Processing CreateTablet for tablet c9e7048a049544ebbbcbd2c184a45e5f (DEFAULT_TABLE table=test-table [id=659f35b1ad6143ca88c1877bcd65ec6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:08.285641 21071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e7048a049544ebbbcbd2c184a45e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:08.286331 21508 ts_tablet_manager.cc:1918] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:08.286536 21508 log.cc:1196] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/faa8b8fd92f24e869e99a56dad7877f5
I20260809 06:26:08.287086 21508 ts_tablet_manager.cc:1939] T faa8b8fd92f24e869e99a56dad7877f5 P 194b90f12683478ba4a0e39cc1bc6d96: Deleting consensus metadata
I20260809 06:26:08.288126 20873 tablet_service.cc:1467] Processing CreateTablet for tablet c9e7048a049544ebbbcbd2c184a45e5f (DEFAULT_TABLE table=test-table [id=659f35b1ad6143ca88c1877bcd65ec6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:08.288645 20873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e7048a049544ebbbcbd2c184a45e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:08.288733 21507 ts_tablet_manager.cc:1905] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:08.291298 21418 tablet_bootstrap.cc:492] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: Bootstrap starting.
I20260809 06:26:08.292894 21418 tablet_bootstrap.cc:654] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:08.296414 21507 ts_tablet_manager.cc:1918] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:08.297626 21503 ts_tablet_manager.cc:1905] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:08.305497 21503 ts_tablet_manager.cc:1918] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:08.305691 21503 log.cc:1196] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/faa8b8fd92f24e869e99a56dad7877f5
I20260809 06:26:08.306265 21503 ts_tablet_manager.cc:1939] T faa8b8fd92f24e869e99a56dad7877f5 P b047aac8e3714d96a76439e4139b6726: Deleting consensus metadata
I20260809 06:26:08.296584 21507 log.cc:1196] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/faa8b8fd92f24e869e99a56dad7877f5
I20260809 06:26:08.307554 21507 ts_tablet_manager.cc:1939] T faa8b8fd92f24e869e99a56dad7877f5 P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting consensus metadata
I20260809 06:26:08.307936 21417 tablet_bootstrap.cc:492] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: Bootstrap starting.
I20260809 06:26:08.308923 20640 catalog_manager.cc:4841] TS 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645): tablet faa8b8fd92f24e869e99a56dad7877f5 (table test-table [id=a6b6f2de222a4c7b932d6a1da9a8f3ca]) successfully deleted
I20260809 06:26:08.309988 20644 catalog_manager.cc:4841] TS 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3:33689): tablet faa8b8fd92f24e869e99a56dad7877f5 (table test-table [id=a6b6f2de222a4c7b932d6a1da9a8f3ca]) successfully deleted
I20260809 06:26:08.310118 21417 tablet_bootstrap.cc:654] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:08.311417 21414 tablet_bootstrap.cc:492] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Bootstrap starting.
I20260809 06:26:08.311681 21418 tablet_bootstrap.cc:492] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: No bootstrap required, opened a new log
I20260809 06:26:08.311932 21418 ts_tablet_manager.cc:1397] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.006s
I20260809 06:26:08.312840 21418 raft_consensus.cc:348] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.313179 21418 raft_consensus.cc:374] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:08.313318 21418 raft_consensus.cc:729] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194b90f12683478ba4a0e39cc1bc6d96, State: Initialized, Role: FOLLOWER
I20260809 06:26:08.313575 20640 catalog_manager.cc:4841] TS b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815): tablet faa8b8fd92f24e869e99a56dad7877f5 (table test-table [id=a6b6f2de222a4c7b932d6a1da9a8f3ca]) successfully deleted
I20260809 06:26:08.313617 21414 tablet_bootstrap.cc:654] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:08.313666 21418 consensus_queue.cc:260] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.314422 21418 ts_tablet_manager.cc:1428] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 06:26:08.321537 21414 tablet_bootstrap.cc:492] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: No bootstrap required, opened a new log
I20260809 06:26:08.321738 21414 ts_tablet_manager.cc:1397] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260809 06:26:08.322362 21417 tablet_bootstrap.cc:492] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: No bootstrap required, opened a new log
I20260809 06:26:08.322561 21414 raft_consensus.cc:348] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.322683 21417 ts_tablet_manager.cc:1397] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.009s
I20260809 06:26:08.322780 21414 raft_consensus.cc:374] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:08.322863 21414 raft_consensus.cc:729] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8226868cd3dc40f19e8b3cb9cf5d4a3e, State: Initialized, Role: FOLLOWER
I20260809 06:26:08.323119 21414 consensus_queue.cc:260] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.323632 21414 ts_tablet_manager.cc:1428] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 06:26:08.323696 21417 raft_consensus.cc:348] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.324005 21417 raft_consensus.cc:374] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:08.324102 21417 raft_consensus.cc:729] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b047aac8e3714d96a76439e4139b6726, State: Initialized, Role: FOLLOWER
I20260809 06:26:08.324473 21417 consensus_queue.cc:260] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.326442 21417 ts_tablet_manager.cc:1428] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 06:26:08.593021 21423 raft_consensus.cc:482] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:26:08.593246 21423 raft_consensus.cc:504] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.594949 21423 leader_election.cc:288] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815), 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3:33689)
I20260809 06:26:08.602385 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.595611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:58357 (local address 127.8.131.1:35815)
0809 06:26:08.596552 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.596558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.596579 (+ 21us) server_negotiation.cc:407] Connection header received
0809 06:26:08.596663 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.596668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.596744 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.596857 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:08.597565 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.598326 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.600296 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.601375 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.601517 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.601619 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:08.601948 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 06:26:08.602097 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":691,"threads_started":1}
I20260809 06:26:08.602387 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.595198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35815 (local address 127.8.131.2:58357)
0809 06:26:08.595902 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:08.595922 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:08.596060 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:08.596901 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:08.596905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:08.596931 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:08.597359 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:08.597378 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.598487 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.598492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:08.600092 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:08.600128 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.601582 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.601590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:08.601767 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.601809 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 06:26:08.601911 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 06:26:08.602050 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":667,"threads_started":1}
I20260809 06:26:08.602723 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.595598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:33605 (local address 127.8.131.3:33689)
0809 06:26:08.595851 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.595857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.596669 (+ 812us) server_negotiation.cc:407] Connection header received
0809 06:26:08.596859 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.596867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.596958 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.597064 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:08.597752 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.598466 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.600598 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.601689 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.601823 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.601948 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:08.602349 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 06:26:08.602504 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 06:26:08.602916 21546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.595326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:33689 (local address 127.8.131.2:33605)
0809 06:26:08.596557 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:08.596574 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:08.596698 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:08.597082 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:08.597085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:08.597106 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:08.597570 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:08.597584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.598617 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.598621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:08.600400 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:08.600414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.601925 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.601931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:08.602163 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.602197 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:26:08.602305 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 06:26:08.602442 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"thread_start_us":868,"threads_started":1}
I20260809 06:26:08.603493 20899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e7048a049544ebbbcbd2c184a45e5f" candidate_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b047aac8e3714d96a76439e4139b6726" is_pre_election: true
I20260809 06:26:08.603631 21310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e7048a049544ebbbcbd2c184a45e5f" candidate_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "194b90f12683478ba4a0e39cc1bc6d96" is_pre_election: true
I20260809 06:26:08.603798 20899 raft_consensus.cc:2454] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8226868cd3dc40f19e8b3cb9cf5d4a3e in term 0.
I20260809 06:26:08.603910 21310 raft_consensus.cc:2454] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8226868cd3dc40f19e8b3cb9cf5d4a3e in term 0.
I20260809 06:26:08.604391 20995 leader_election.cc:302] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 194b90f12683478ba4a0e39cc1bc6d96, 8226868cd3dc40f19e8b3cb9cf5d4a3e, b047aac8e3714d96a76439e4139b6726; no voters:
I20260809 06:26:08.604813 21423 raft_consensus.cc:2781] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:26:08.604946 21423 raft_consensus.cc:482] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:26:08.605052 21423 raft_consensus.cc:3037] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:08.607190 21423 raft_consensus.cc:504] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.607812 21423 leader_election.cc:288] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [CANDIDATE]: Term 1 election: Requested vote from peers b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815), 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3:33689)
I20260809 06:26:08.608186 20899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e7048a049544ebbbcbd2c184a45e5f" candidate_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b047aac8e3714d96a76439e4139b6726"
I20260809 06:26:08.608430 20899 raft_consensus.cc:3037] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:08.608336 21310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e7048a049544ebbbcbd2c184a45e5f" candidate_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "194b90f12683478ba4a0e39cc1bc6d96"
I20260809 06:26:08.608606 21310 raft_consensus.cc:3037] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:08.610414 20899 raft_consensus.cc:2454] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8226868cd3dc40f19e8b3cb9cf5d4a3e in term 1.
I20260809 06:26:08.610544 21310 raft_consensus.cc:2454] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8226868cd3dc40f19e8b3cb9cf5d4a3e in term 1.
I20260809 06:26:08.610841 20995 leader_election.cc:302] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8226868cd3dc40f19e8b3cb9cf5d4a3e, b047aac8e3714d96a76439e4139b6726; no voters:
I20260809 06:26:08.611119 21423 raft_consensus.cc:2781] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:08.611534 21423 raft_consensus.cc:686] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 LEADER]: Becoming Leader. State: Replica: 8226868cd3dc40f19e8b3cb9cf5d4a3e, State: Running, Role: LEADER
I20260809 06:26:08.611934 21423 consensus_queue.cc:237] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } }
I20260809 06:26:08.615370 20663 catalog_manager.cc:5495] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2). New cstate: current_term: 1 leader_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8226868cd3dc40f19e8b3cb9cf5d4a3e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 } health_report { overall_health: UNKNOWN } } }
I20260809 06:26:08.690346 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.684871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56472 (local address 127.8.131.2:34645)
0809 06:26:08.685457 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.685461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.685480 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:26:08.685562 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.685565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.685652 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.685828 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:08.686191 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.687013 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.688281 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.688681 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.688785 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.689187 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:08.689762 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:08.689895 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:08.690001 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 06:26:08.690076 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":323,"threads_started":1}
I20260809 06:26:08.705322 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.700043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:33419 (local address 127.8.131.1:35815)
0809 06:26:08.700264 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.700270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.700857 (+ 587us) server_negotiation.cc:407] Connection header received
0809 06:26:08.701036 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.701045 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.701117 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.701195 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:08.701644 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.702356 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.703627 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.704204 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.704311 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.704692 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:08.704828 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:08.704899 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:08.705031 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 06:26:08.705097 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 06:26:08.705379 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.699978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35815 (local address 127.8.131.2:33419)
0809 06:26:08.700767 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:08.700783 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:08.700904 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:08.701219 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:08.701222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:08.701231 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:26:08.701503 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:08.701510 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.702467 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.702470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:08.703508 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:08.703518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.704361 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.704366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:08.704487 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.704539 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:26:08.704925 (+ 386us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:26:08.704933 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 06:26:08.704993 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 06:26:08.705096 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":393,"threads_started":1}
I20260809 06:26:08.714175 21310 raft_consensus.cc:1261] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Refusing update from remote peer 8226868cd3dc40f19e8b3cb9cf5d4a3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:08.714284 20899 raft_consensus.cc:1261] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Refusing update from remote peer 8226868cd3dc40f19e8b3cb9cf5d4a3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:08.715171 21423 consensus_queue.cc:1035] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b047aac8e3714d96a76439e4139b6726" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:08.715494 21548 consensus_queue.cc:1035] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "194b90f12683478ba4a0e39cc1bc6d96" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 33689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:08.742975 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.737608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:49385 (local address 127.8.131.2:34645)
0809 06:26:08.737777 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.737781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.737895 (+ 114us) server_negotiation.cc:407] Connection header received
0809 06:26:08.738016 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.738020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.738080 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.738151 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:08.738535 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.739511 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.741240 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.741663 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.741795 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.742203 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:08.742298 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:08.742436 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:08.742586 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 06:26:08.742663 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 06:26:08.743046 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.737217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:34645 (local address 127.8.131.1:49385)
0809 06:26:08.737808 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:08.737824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:08.737934 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:08.738181 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:08.738184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:08.738192 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 06:26:08.738413 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:08.738420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.739631 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.739662 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:08.741096 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:08.741110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.741832 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.741840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:08.741991 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.742028 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 06:26:08.742450 (+ 422us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 06:26:08.742463 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 06:26:08.742564 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 06:26:08.742693 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":309,"threads_started":1}
I20260809 06:26:08.775769 20665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:08.776057 20665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:08.779173 20665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61c2e9b2009c482e8684c44ec3602564: Sending DeleteTablet for 3 replicas of tablet c9e7048a049544ebbbcbd2c184a45e5f
I20260809 06:26:08.780077 21071 tablet_service.cc:1514] Processing DeleteTablet for tablet c9e7048a049544ebbbcbd2c184a45e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:08 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:56458
I20260809 06:26:08.780417 21279 tablet_service.cc:1514] Processing DeleteTablet for tablet c9e7048a049544ebbbcbd2c184a45e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:08 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:35088
I20260809 06:26:08.780740 21507 tablet_replica.cc:330] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: stopping tablet replica
I20260809 06:26:08.780938 21507 raft_consensus.cc:2229] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:26:08.781445 21507 raft_consensus.cc:2258] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:08.781795 21508 tablet_replica.cc:330] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: stopping tablet replica
I20260809 06:26:08.781985 21508 raft_consensus.cc:2229] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:08.782191 21508 raft_consensus.cc:2258] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:08.783424 21508 ts_tablet_manager.cc:1905] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:08.784196 21507 ts_tablet_manager.cc:1905] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:08.788043 21508 ts_tablet_manager.cc:1918] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:26:08.788137 21508 log.cc:1196] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/c9e7048a049544ebbbcbd2c184a45e5f
I20260809 06:26:08.788537 21508 ts_tablet_manager.cc:1939] T c9e7048a049544ebbbcbd2c184a45e5f P 194b90f12683478ba4a0e39cc1bc6d96: Deleting consensus metadata
I20260809 06:26:08.789474 20644 catalog_manager.cc:4841] TS 194b90f12683478ba4a0e39cc1bc6d96 (127.8.131.3:33689): tablet c9e7048a049544ebbbcbd2c184a45e5f (table test-table [id=659f35b1ad6143ca88c1877bcd65ec6c]) successfully deleted
I20260809 06:26:08.789544 20873 tablet_service.cc:1514] Processing DeleteTablet for tablet c9e7048a049544ebbbcbd2c184a45e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:08 UTC) from {username='oryx', principal='oryx/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:45150
I20260809 06:26:08.789867 21503 tablet_replica.cc:330] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: stopping tablet replica
I20260809 06:26:08.790036 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 20767
I20260809 06:26:08.790062 21503 raft_consensus.cc:2229] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:08.790269 21503 raft_consensus.cc:2258] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:08.792631 21503 ts_tablet_manager.cc:1905] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:08.792788 21507 ts_tablet_manager.cc:1918] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:26:08.792955 21507 log.cc:1196] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/c9e7048a049544ebbbcbd2c184a45e5f
I20260809 06:26:08.793493 21507 ts_tablet_manager.cc:1939] T c9e7048a049544ebbbcbd2c184a45e5f P 8226868cd3dc40f19e8b3cb9cf5d4a3e: Deleting consensus metadata
I20260809 06:26:08.794723 20640 catalog_manager.cc:4841] TS 8226868cd3dc40f19e8b3cb9cf5d4a3e (127.8.131.2:34645): tablet c9e7048a049544ebbbcbd2c184a45e5f (table test-table [id=659f35b1ad6143ca88c1877bcd65ec6c]) successfully deleted
I20260809 06:26:08.799678 21503 ts_tablet_manager.cc:1918] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 06:26:08.799827 21503 log.cc:1196] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/c9e7048a049544ebbbcbd2c184a45e5f
I20260809 06:26:08.800289 21503 ts_tablet_manager.cc:1939] T c9e7048a049544ebbbcbd2c184a45e5f P b047aac8e3714d96a76439e4139b6726: Deleting consensus metadata
Aug 09 06:26:08 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.1@KRBTEST.COM
I20260809 06:26:08.801445 20640 catalog_manager.cc:4841] TS b047aac8e3714d96a76439e4139b6726 (127.8.131.1:35815): tablet c9e7048a049544ebbbcbd2c184a45e5f (table test-table [id=659f35b1ad6143ca88c1877bcd65ec6c]) successfully deleted
I20260809 06:26:08.808055 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.791829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.8.131.1:35815)
0809 06:26:08.792048 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.792056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.792081 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:26:08.792390 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.792396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.792462 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.792578 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:08.794569 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.795454 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.796904 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.797367 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.797535 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.802128 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:08.802154 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:08.802170 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:08.802202 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:08.805184 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:08.805913 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:08.805921 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:08.805923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:08.806052 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:08.806457 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:08.806472 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:08.806474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:08.806835 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:08.807153 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:08.807462 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 06:26:08.807721 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 06:26:08.808741 20963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:08.943117 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 20767
I20260809 06:26:08.982402 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 20973
Aug 09 06:26:08 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.2@KRBTEST.COM
I20260809 06:26:08.998818 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:08.983303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56484 (local address 127.8.131.2:34645)
0809 06:26:08.983546 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:08.983552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:08.983574 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:26:08.983696 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:08.983701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:08.983805 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:08.983943 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:08.985174 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.985968 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.987240 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:08.987794 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:08.987951 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:08.992162 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:08.992177 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:08.992199 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:08.992233 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:08.995597 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:08.996405 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:08.996416 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:08.996420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:08.996600 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:08.996988 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:08.996993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:08.996997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:08.997460 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:08.997826 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:08.998128 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 06:26:08.998374 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 06:26:08.999605 21160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:09.123150 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 20973
I20260809 06:26:09.151554 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 21177
Aug 09 06:26:09 dist-test-slave-mjjr krb5kdc[20601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.131.3@KRBTEST.COM
I20260809 06:26:09.166814 21576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:09.152406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35132 (local address 127.8.131.3:33689)
0809 06:26:09.152912 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:09.152916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:09.152930 (+ 14us) server_negotiation.cc:407] Connection header received
0809 06:26:09.152982 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:09.152985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:09.153042 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:09.153161 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:09.154517 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:09.155211 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:09.156585 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:09.157003 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:09.157141 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:09.161328 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:09.161343 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:09.161359 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:09.161401 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:09.164129 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:09.164770 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:09.164778 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:09.164780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:09.164926 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:09.165298 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:09.165304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:09.165307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:09.165645 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:09.166005 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:09.166229 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 06:26:09.166477 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":241,"threads_started":1}
I20260809 06:26:09.167488 21375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:09.293417 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 21177
I20260809 06:26:09.323391 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 20619
I20260809 06:26:09.324783 20724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:09.440586 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 20619
2026-08-09T06:26:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6307 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:26:09 dist-test-slave-mjjr krb5kdc[21585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:26:09 dist-test-slave-mjjr krb5kdc[21585](info): set up 2 sockets
Aug 09 06:26:09 dist-test-slave-mjjr krb5kdc[21585](info): commencing operation
krb5kdc: starting...
W20260809 06:26:10.758253 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:26:10 dist-test-slave-mjjr krb5kdc[21585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:26:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:26:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:26:11.012007 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:37541
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:37541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:11.339722 21604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:11.340070 21604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:11.340253 21604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:11.347462 21604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:26:11.347609 21604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:11.347708 21604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:11.347819 21604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:26:11.347900 21604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:26:11.357784 21604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:37541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:37541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:11.359084 21604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:11.360417 21604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:11.372273 21612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:11.372243 21610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:11.372308 21609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:11.373415 21604 server_base.cc:1034] running on GCE node
I20260809 06:26:11.374145 21604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:11.375466 21604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:11.376706 21604 hybrid_clock.cc:648] HybridClock initialized: now 1786256771376658 us; error 56 us; skew 500 ppm
I20260809 06:26:11.377067 21604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T06:26:11Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2006 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:26:11 dist-test-slave-mjjr krb5kdc[21619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:26:11 dist-test-slave-mjjr krb5kdc[21619](info): set up 2 sockets
Aug 09 06:26:11 dist-test-slave-mjjr krb5kdc[21619](info): commencing operation
krb5kdc: starting...
W20260809 06:26:12.755865 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.006s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:26:12 dist-test-slave-mjjr krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:26:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:26:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:26:13.012530 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45267
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:45267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:13.346150 21642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:13.346439 21642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:13.346666 21642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:13.354473 21642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:26:13.354594 21642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:13.354642 21642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:13.354693 21642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:26:13.354803 21642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:26:13.365507 21642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:45267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:13.366874 21642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:13.368284 21642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:13.380605 21648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:13.380568 21651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:13.380673 21649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:13.381547 21642 server_base.cc:1034] running on GCE node
I20260809 06:26:13.382318 21642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:13.383941 21642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:13.385187 21642 hybrid_clock.cc:648] HybridClock initialized: now 1786256773385148 us; error 51 us; skew 500 ppm
I20260809 06:26:13.385576 21642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:13 dist-test-slave-mjjr krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:13.393392 21642 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:26:13.394984 21642 webserver.cc:460] Webserver started at http://127.8.131.62:39589/ using document root <none> and password file <none>
I20260809 06:26:13.395761 21642 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:13.395910 21642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:13.396234 21642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:13.399281 21642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "863817f25cdf4c8ca2e04dea1d51249f"
format_stamp: "Formatted at 2026-08-09 06:26:13 on dist-test-slave-mjjr"
server_key: "2e6953449251333d283ed91129906bc2"
server_key_iv: "10dd79a93fd93edc1e9136b79a598092"
server_key_version: "encryptionkey@0"
I20260809 06:26:13.400126 21642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "863817f25cdf4c8ca2e04dea1d51249f"
format_stamp: "Formatted at 2026-08-09 06:26:13 on dist-test-slave-mjjr"
server_key: "2e6953449251333d283ed91129906bc2"
server_key_iv: "10dd79a93fd93edc1e9136b79a598092"
server_key_version: "encryptionkey@0"
I20260809 06:26:13.405246 21642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:26:13.408675 21663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:13.409916 21642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:26:13.410112 21642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "863817f25cdf4c8ca2e04dea1d51249f"
format_stamp: "Formatted at 2026-08-09 06:26:13 on dist-test-slave-mjjr"
server_key: "2e6953449251333d283ed91129906bc2"
server_key_iv: "10dd79a93fd93edc1e9136b79a598092"
server_key_version: "encryptionkey@0"
I20260809 06:26:13.410329 21642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:13.444936 21642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:13.448994 21642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:13.449292 21642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:13.467913 21642 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:45267
I20260809 06:26:13.467995 21738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:45267 every 8 connection(s)
I20260809 06:26:13.469269 21642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:26:13.472873 21739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:13.478325 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 21642
I20260809 06:26:13.478555 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:26:13.479341 8716 external_mini_cluster.cc:1428] Setting key 0443796eb87b19170214f33b03ba41e8
I20260809 06:26:13.484186 21739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: Bootstrap starting.
I20260809 06:26:13.487880 21739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:13.488902 21739 log.cc:824] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:13.491979 21739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: No bootstrap required, opened a new log
Aug 09 06:26:13 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:26:13.502597 21739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } }
I20260809 06:26:13.503070 21739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:13.503180 21739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863817f25cdf4c8ca2e04dea1d51249f, State: Initialized, Role: FOLLOWER
I20260809 06:26:13.503878 21739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } }
I20260809 06:26:13.504209 21739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:26:13.504411 21739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:26:13.504601 21739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:13.506023 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:13.481675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42922 (local address 127.8.131.62:45267)
0809 06:26:13.482672 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:13.482683 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:13.482730 (+ 47us) server_negotiation.cc:407] Connection header received
0809 06:26:13.484734 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:13.484757 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:13.485128 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:13.485516 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:13.487402 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:13.489146 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:13.490720 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:13.491461 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:13.491611 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:13.496873 (+ 5262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:13.496883 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:13.496904 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:13.496968 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:13.501306 (+ 4338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:13.501968 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:13.501977 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:13.501986 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:13.502203 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:13.502607 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:13.502614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:13.502621 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:13.503329 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:13.503672 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:13.504159 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 06:26:13.504660 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":482,"threads_started":1}
I20260809 06:26:13.507465 21739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } }
I20260809 06:26:13.508011 21739 leader_election.cc:302] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 863817f25cdf4c8ca2e04dea1d51249f; no voters:
I20260809 06:26:13.508596 21739 leader_election.cc:288] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:26:13.508752 21746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:13.509717 21746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 1 LEADER]: Becoming Leader. State: Replica: 863817f25cdf4c8ca2e04dea1d51249f, State: Running, Role: LEADER
I20260809 06:26:13.510215 21746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } }
I20260809 06:26:13.510664 21739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:26:13.512972 21752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863817f25cdf4c8ca2e04dea1d51249f. Latest consensus state: current_term: 1 leader_uuid: "863817f25cdf4c8ca2e04dea1d51249f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } } }
I20260809 06:26:13.513058 21747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "863817f25cdf4c8ca2e04dea1d51249f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } } }
I20260809 06:26:13.513396 21752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:13.513480 21747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:13.516795 21758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:26:13.522406 21758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:26:13.531072 21758 catalog_manager.cc:1347] Generated new cluster ID: b64e3e6b47654a8c982331547249a01e
I20260809 06:26:13.531176 21758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:26:13.543313 21758 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:26:13.545179 21758 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:26:13.552842 21758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: Generated new TSK 0
I20260809 06:26:13.553607 21758 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:26:13.632944 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45267
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:13.999024 21773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:13.999276 21773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:13.999367 21773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:14.007226 21773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:14.007335 21773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:14.007444 21773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:26:14.017308 21773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:14.018471 21773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:14.019789 21773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:14.032395 21778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:14.032435 21780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:14.032472 21782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:14.033224 21773 server_base.cc:1034] running on GCE node
I20260809 06:26:14.033820 21773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:14.034710 21773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:14.035936 21773 hybrid_clock.cc:648] HybridClock initialized: now 1786256774035898 us; error 49 us; skew 500 ppm
I20260809 06:26:14.036247 21773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:14 dist-test-slave-mjjr krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:14.039448 21773 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:26:14.040936 21773 webserver.cc:460] Webserver started at http://127.8.131.1:36187/ using document root <none> and password file <none>
I20260809 06:26:14.041615 21773 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:14.041761 21773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:14.042073 21773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:14.045068 21773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "7a8e33ce303c4d8abee7b85d4c1ca4f0"
format_stamp: "Formatted at 2026-08-09 06:26:14 on dist-test-slave-mjjr"
server_key: "80b89220c952cbf0d330aa3b70de77b8"
server_key_iv: "e174e5559161bf8b933de361cff0627d"
server_key_version: "encryptionkey@0"
I20260809 06:26:14.045879 21773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "7a8e33ce303c4d8abee7b85d4c1ca4f0"
format_stamp: "Formatted at 2026-08-09 06:26:14 on dist-test-slave-mjjr"
server_key: "80b89220c952cbf0d330aa3b70de77b8"
server_key_iv: "e174e5559161bf8b933de361cff0627d"
server_key_version: "encryptionkey@0"
I20260809 06:26:14.051038 21773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 06:26:14.054479 21795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:14.055882 21773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:26:14.056057 21773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "7a8e33ce303c4d8abee7b85d4c1ca4f0"
format_stamp: "Formatted at 2026-08-09 06:26:14 on dist-test-slave-mjjr"
server_key: "80b89220c952cbf0d330aa3b70de77b8"
server_key_iv: "e174e5559161bf8b933de361cff0627d"
server_key_version: "encryptionkey@0"
I20260809 06:26:14.056258 21773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:14.071517 21773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:14.075456 21773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:14.075815 21773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:14.076763 21773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:14.078121 21773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:14.078249 21773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:14.078413 21773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:14.078506 21773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:14.112437 21773 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:43931
I20260809 06:26:14.112475 21951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:43931 every 8 connection(s)
I20260809 06:26:14.113754 21773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:26:14.121912 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 21773
I20260809 06:26:14.122116 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:26:14.122766 8716 external_mini_cluster.cc:1428] Setting key aa92b80ae378e1daf91a80115af45d92
Aug 09 06:26:14 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:26:14.135401 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:14.116453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:47231 (local address 127.8.131.62:45267)
0809 06:26:14.116936 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:14.116941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:14.117928 (+ 987us) server_negotiation.cc:407] Connection header received
0809 06:26:14.119715 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:14.119721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:14.119816 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:14.119938 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:14.121410 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:14.122248 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:14.124093 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:14.124605 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:14.124726 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:14.128497 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:14.128506 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:14.128514 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:14.128559 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:14.132211 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:14.132967 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:14.132976 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:14.132982 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:14.133138 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:14.133568 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:14.133576 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:14.133581 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:14.133972 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:14.134133 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:14.134868 (+ 735us) server_negotiation.cc:299] Negotiation successful
0809 06:26:14.135121 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":382,"thread_start_us":268,"threads_started":1}
I20260809 06:26:14.136749 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:14.117038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:47231)
0809 06:26:14.117753 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:14.117800 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:14.119420 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:14.120162 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:14.120171 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:14.120676 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:14.121217 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:14.121233 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:14.122417 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:14.122421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:14.123955 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:14.123970 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:14.124852 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:14.124861 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:14.125104 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:14.125803 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:14.125825 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:14.128265 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:14.132359 (+ 4094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:14.132365 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:14.132377 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:14.132836 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:14.133256 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:14.133260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:14.133262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:14.133453 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:14.134098 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:14.134104 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:14.134434 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 06:26:14.134665 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 06:26:14.135228 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":287,"threads_started":1}
I20260809 06:26:14.138298 21953 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45267
I20260809 06:26:14.138608 21953 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:14.139317 21953 heartbeater.cc:511] Master 127.8.131.62:45267 requested a full tablet report, sending...
I20260809 06:26:14.141479 21687 ts_manager.cc:194] Registered new tserver with Master: 7a8e33ce303c4d8abee7b85d4c1ca4f0 (127.8.131.1:43931)
I20260809 06:26:14.143047 21687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:47231
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:26:14.178402 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45267
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:14.500698 21960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:14.501037 21960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:14.501210 21960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:14.508467 21960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:14.508601 21960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:14.508777 21960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:26:14.518471 21960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:14.519780 21960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:14.521095 21960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:14.534107 21970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:14.534124 21967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:14.534157 21966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:14.534938 21960 server_base.cc:1034] running on GCE node
I20260809 06:26:14.535553 21960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:14.536463 21960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:14.537686 21960 hybrid_clock.cc:648] HybridClock initialized: now 1786256774537635 us; error 58 us; skew 500 ppm
I20260809 06:26:14.537993 21960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:14 dist-test-slave-mjjr krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:14.541198 21960 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:26:14.542519 21960 webserver.cc:460] Webserver started at http://127.8.131.2:38139/ using document root <none> and password file <none>
I20260809 06:26:14.543138 21960 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:14.543233 21960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:14.543488 21960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:14.546271 21960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "5f952435737342e2949339049e9c0aff"
format_stamp: "Formatted at 2026-08-09 06:26:14 on dist-test-slave-mjjr"
server_key: "debdfbcf2234037109e2ee3d3c48ea60"
server_key_iv: "81bdaa9c65c595f6e8ae51bb2e475c64"
server_key_version: "encryptionkey@0"
I20260809 06:26:14.547011 21960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "5f952435737342e2949339049e9c0aff"
format_stamp: "Formatted at 2026-08-09 06:26:14 on dist-test-slave-mjjr"
server_key: "debdfbcf2234037109e2ee3d3c48ea60"
server_key_iv: "81bdaa9c65c595f6e8ae51bb2e475c64"
server_key_version: "encryptionkey@0"
I20260809 06:26:14.552063 21960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 06:26:14.555516 21983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:14.556986 21960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:26:14.557161 21960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "5f952435737342e2949339049e9c0aff"
format_stamp: "Formatted at 2026-08-09 06:26:14 on dist-test-slave-mjjr"
server_key: "debdfbcf2234037109e2ee3d3c48ea60"
server_key_iv: "81bdaa9c65c595f6e8ae51bb2e475c64"
server_key_version: "encryptionkey@0"
I20260809 06:26:14.557369 21960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:14.579067 21960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:14.582816 21960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:14.583060 21960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:14.584048 21960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:14.585314 21960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:14.585390 21960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:14.585548 21960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:14.585598 21960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:14.618677 21960 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:33515
I20260809 06:26:14.618767 22136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:33515 every 8 connection(s)
I20260809 06:26:14.620075 21960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:26:14.620399 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 21960
I20260809 06:26:14.620584 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:26:14.621243 8716 external_mini_cluster.cc:1428] Setting key f497d1e5081e295b23c8c4171662c04a
Aug 09 06:26:14 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:26:14.642156 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:14.622950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:34055 (local address 127.8.131.62:45267)
0809 06:26:14.623181 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:14.623205 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:14.624555 (+ 1350us) server_negotiation.cc:407] Connection header received
0809 06:26:14.626407 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:14.626413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:14.626517 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:14.626636 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:14.628242 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:14.628979 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:14.630515 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:14.630969 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:14.631108 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:14.634896 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:14.634905 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:14.634912 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:14.634957 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:14.638696 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:14.639447 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:14.639455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:14.639462 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:14.639631 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:14.640092 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:14.640100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:14.640106 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:14.640543 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:14.640701 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:14.641377 (+ 676us) server_negotiation.cc:299] Negotiation successful
0809 06:26:14.641686 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 06:26:14.642786 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:14.623569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:34055)
0809 06:26:14.624398 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:14.624440 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:14.626136 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:14.626866 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:14.626874 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:14.627412 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:14.628068 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:14.628085 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:14.629129 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:14.629134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:14.630361 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:14.630372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:14.631200 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:14.631207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:14.631441 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:14.632182 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:14.632204 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:14.634648 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:14.638859 (+ 4211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:14.638865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:14.638877 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:14.639304 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:14.639764 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:14.639767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:14.639770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:14.639964 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:14.640672 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:14.640678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:14.641003 (+ 325us) client_negotiation.cc:769] Sending connection context
0809 06:26:14.641228 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 06:26:14.641656 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":369,"threads_started":1}
I20260809 06:26:14.644428 22137 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45267
I20260809 06:26:14.644742 22137 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:14.645530 22137 heartbeater.cc:511] Master 127.8.131.62:45267 requested a full tablet report, sending...
I20260809 06:26:14.647056 21687 ts_manager.cc:194] Registered new tserver with Master: 5f952435737342e2949339049e9c0aff (127.8.131.2:33515)
I20260809 06:26:14.647971 21687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:34055
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:26:14.680708 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45267
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:15.013854 22144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:15.014179 22144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:15.014380 22144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:15.021756 22144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:15.021908 22144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:15.022083 22144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:26:15.031808 22144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:45267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:15.033191 22144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:15.034518 22144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:15.047839 22152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:15.047884 22154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:15.047791 22157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:15.048985 22144 server_base.cc:1034] running on GCE node
I20260809 06:26:15.049561 22144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:15.050453 22144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:15.051741 22144 hybrid_clock.cc:648] HybridClock initialized: now 1786256775051685 us; error 45 us; skew 500 ppm
I20260809 06:26:15.052068 22144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:15 dist-test-slave-mjjr krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256775, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:15.055246 22144 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:26:15.056783 22144 webserver.cc:460] Webserver started at http://127.8.131.3:34707/ using document root <none> and password file <none>
I20260809 06:26:15.057492 22144 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:15.057663 22144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:15.057991 22144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:15.061129 22144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "66d214b8751f42888b63a314240df4c6"
format_stamp: "Formatted at 2026-08-09 06:26:15 on dist-test-slave-mjjr"
server_key: "312415be6f2ebe26208bf7cd75b7262c"
server_key_iv: "9e7d99736769ef9861802bd6d7b2e03a"
server_key_version: "encryptionkey@0"
I20260809 06:26:15.061986 22144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "66d214b8751f42888b63a314240df4c6"
format_stamp: "Formatted at 2026-08-09 06:26:15 on dist-test-slave-mjjr"
server_key: "312415be6f2ebe26208bf7cd75b7262c"
server_key_iv: "9e7d99736769ef9861802bd6d7b2e03a"
server_key_version: "encryptionkey@0"
I20260809 06:26:15.067595 22144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 06:26:15.071031 22166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:15.072331 22144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:26:15.072525 22144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "66d214b8751f42888b63a314240df4c6"
format_stamp: "Formatted at 2026-08-09 06:26:15 on dist-test-slave-mjjr"
server_key: "312415be6f2ebe26208bf7cd75b7262c"
server_key_iv: "9e7d99736769ef9861802bd6d7b2e03a"
server_key_version: "encryptionkey@0"
I20260809 06:26:15.072722 22144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:15.100437 22144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:15.104573 22144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:15.104874 22144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:15.105877 22144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:15.107292 22144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:15.107400 22144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:15.107559 22144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:15.107632 22144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:15.143366 22144 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:40019
I20260809 06:26:15.143509 22340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:40019 every 8 connection(s)
I20260809 06:26:15.144732 22144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:26:15.145974 21953 heartbeater.cc:503] Master 127.8.131.62:45267 was elected leader, sending a full tablet report...
I20260809 06:26:15.155100 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 22144
I20260809 06:26:15.155274 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:26:15.155882 8716 external_mini_cluster.cc:1428] Setting key 1b0e3f944504940c0aa1dde75f9d0c06
Aug 09 06:26:15 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256775, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:26:15.165143 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:15.147375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:60025 (local address 127.8.131.62:45267)
0809 06:26:15.147900 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:15.147907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:15.148900 (+ 993us) server_negotiation.cc:407] Connection header received
0809 06:26:15.150661 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:15.150665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:15.150729 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:15.150824 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:15.152405 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.153179 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.154742 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.155146 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.155300 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:15.159297 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:15.159302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:15.159305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:15.159336 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:15.162215 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:15.162979 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:15.162983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:15.162985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:15.163106 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:15.163546 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:15.163549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:15.163551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:15.163881 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:15.163992 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:15.164674 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 06:26:15.164861 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":297,"threads_started":1}
I20260809 06:26:15.166030 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:15.148030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:60025)
0809 06:26:15.148733 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:15.148777 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:15.150418 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:15.151027 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:15.151035 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:15.151567 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:15.152213 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:15.152231 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.153341 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.153345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:15.154570 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:15.154582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.155429 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.155438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:15.155769 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:15.156521 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:15.156542 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:15.159088 (+ 2546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:15.162357 (+ 3269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:15.162364 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:15.162375 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:15.162847 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:15.163208 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:15.163212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:15.163214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:15.163439 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:15.164014 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:15.164021 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:15.164334 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 06:26:15.164559 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 06:26:15.164931 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":290,"threads_started":1}
I20260809 06:26:15.167348 22342 heartbeater.cc:347] Connected to a master server at 127.8.131.62:45267
I20260809 06:26:15.167608 22342 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:15.168334 22342 heartbeater.cc:511] Master 127.8.131.62:45267 requested a full tablet report, sending...
I20260809 06:26:15.169651 21687 ts_manager.cc:194] Registered new tserver with Master: 66d214b8751f42888b63a314240df4c6 (127.8.131.3:40019)
I20260809 06:26:15.170818 21687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:60025
I20260809 06:26:15.171574 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab.
Entry for principal oryx/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab.
WARNING: no policy specified for oryx/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab.
Entry for principal oryx/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab.
WARNING: no policy specified for oryx/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab.
Entry for principal oryx/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab.
I20260809 06:26:15.322633 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 21773
Aug 09 06:26:15 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:26:15.339566 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:15.323905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36452 (local address 127.8.131.1:43931)
0809 06:26:15.324779 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:15.324786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:15.324812 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:26:15.324873 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:15.324881 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:15.325091 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:15.325242 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:15.326667 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.327834 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.329043 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.329586 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.329683 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:15.334000 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:15.334011 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:15.334025 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:15.334061 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:15.336735 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:15.337459 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:15.337467 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:15.337469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:15.337589 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:15.337972 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:15.337978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:15.337983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:15.338314 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:15.338637 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:15.338926 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 06:26:15.339200 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":536,"threads_started":1}
I20260809 06:26:15.340821 21946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:15.426283 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 21773
I20260809 06:26:15.450186 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 21960
Aug 09 06:26:15 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:26:15.466460 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:15.451115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.8.131.2:33515)
0809 06:26:15.451727 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:15.451734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:15.451761 (+ 27us) server_negotiation.cc:407] Connection header received
0809 06:26:15.451870 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:15.451881 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:15.452101 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:15.452234 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:15.453404 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.454441 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.455876 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.456451 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.456553 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:15.460683 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:15.460699 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:15.460718 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:15.460780 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:15.463636 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:15.464333 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:15.464343 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:15.464346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:15.464473 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:15.464840 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:15.464846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:15.464850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:15.465184 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:15.465506 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:15.465837 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 06:26:15.466114 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":305,"threads_started":1}
I20260809 06:26:15.467890 22128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:15.553725 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 21960
I20260809 06:26:15.578795 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 22144
Aug 09 06:26:15 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:26:15.594715 22360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:15.579874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60916 (local address 127.8.131.3:40019)
0809 06:26:15.580400 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:15.580408 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:15.580437 (+ 29us) server_negotiation.cc:407] Connection header received
0809 06:26:15.580493 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:15.580499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:15.580704 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:15.580830 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:15.582069 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.583050 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.584438 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:15.584960 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:15.585053 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:15.589323 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:15.589334 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:15.589347 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:15.589378 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:15.592350 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:15.592940 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:15.592947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:15.592949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:15.593060 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:15.593404 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:15.593407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:15.593409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:15.593739 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:15.593960 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:15.594216 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 06:26:15.594427 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":240,"threads_started":1}
I20260809 06:26:15.596247 22331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:15.679682 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 22144
I20260809 06:26:15.702912 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 21642
I20260809 06:26:15.704295 21731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:15.812155 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 21642
I20260809 06:26:15.842294 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45267
--webserver_interface=127.8.131.62
--webserver_port=39589
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:45267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:16.179327 22366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:16.179631 22366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:16.179807 22366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:16.186762 22366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:26:16.186865 22366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:16.186913 22366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:16.186966 22366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:26:16.187012 22366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:26:16.196625 22366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:45267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=39589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:16.197822 22366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:16.199030 22366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:16.210486 22373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:16.210434 22377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:16.210529 22374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:16.211755 22366 server_base.cc:1034] running on GCE node
I20260809 06:26:16.212482 22366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:16.213773 22366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:16.214951 22366 hybrid_clock.cc:648] HybridClock initialized: now 1786256776214897 us; error 64 us; skew 500 ppm
I20260809 06:26:16.215431 22366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:16 dist-test-slave-mjjr krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256776, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:16.218567 22366 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:26:16.220060 22366 webserver.cc:460] Webserver started at http://127.8.131.62:39589/ using document root <none> and password file <none>
I20260809 06:26:16.220692 22366 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:16.220817 22366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:16.226140 22366 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 06:26:16.228909 22389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:16.230191 22366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:26:16.230384 22366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "863817f25cdf4c8ca2e04dea1d51249f"
format_stamp: "Formatted at 2026-08-09 06:26:13 on dist-test-slave-mjjr"
server_key: "2e6953449251333d283ed91129906bc2"
server_key_iv: "10dd79a93fd93edc1e9136b79a598092"
server_key_version: "encryptionkey@0"
I20260809 06:26:16.231343 22366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:16.243095 22366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:16.246978 22366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:16.247255 22366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:16.265720 22366 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:45267
I20260809 06:26:16.265739 22477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:45267 every 8 connection(s)
I20260809 06:26:16.267217 22366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:26:16.273862 22479 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:26:16.277293 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 22366
I20260809 06:26:16.277326 22479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: Bootstrap starting.
I20260809 06:26:16.278327 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:43931
--local_ip_for_outbound_sockets=127.8.131.1
--tserver_master_addrs=127.8.131.62:45267
--webserver_port=36187
--webserver_interface=127.8.131.1
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.131.1
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:26:16.300977 22479 log.cc:824] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:16.310001 22479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:26:16.310587 22479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: Bootstrap complete.
I20260809 06:26:16.320261 22479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } }
I20260809 06:26:16.321103 22479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863817f25cdf4c8ca2e04dea1d51249f, State: Initialized, Role: FOLLOWER
I20260809 06:26:16.321841 22479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } }
I20260809 06:26:16.322090 22479 raft_consensus.cc:388] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:26:16.322242 22479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:26:16.322436 22479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:26:16.326314 22479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } }
I20260809 06:26:16.326869 22479 leader_election.cc:302] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 863817f25cdf4c8ca2e04dea1d51249f; no voters:
I20260809 06:26:16.327528 22479 leader_election.cc:288] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 06:26:16.327690 22486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 06:26:16.329020 22486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [term 2 LEADER]: Becoming Leader. State: Replica: 863817f25cdf4c8ca2e04dea1d51249f, State: Running, Role: LEADER
I20260809 06:26:16.329571 22486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } }
I20260809 06:26:16.329906 22479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:26:16.332195 22488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863817f25cdf4c8ca2e04dea1d51249f. Latest consensus state: current_term: 2 leader_uuid: "863817f25cdf4c8ca2e04dea1d51249f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } } }
I20260809 06:26:16.332636 22488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:16.333140 22487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "863817f25cdf4c8ca2e04dea1d51249f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863817f25cdf4c8ca2e04dea1d51249f" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45267 } } }
I20260809 06:26:16.333447 22487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:16.338157 22493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:26:16.345180 22493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:26:16.346947 22493 catalog_manager.cc:1259] Loaded cluster ID: b64e3e6b47654a8c982331547249a01e
I20260809 06:26:16.347075 22493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:26:16.352499 22493 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:26:16.354358 22493 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 863817f25cdf4c8ca2e04dea1d51249f: Loaded TSK: 0
I20260809 06:26:16.356267 22493 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 06:26:16.641996 22481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:16.642330 22481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:16.642522 22481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:16.650813 22481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:16.650947 22481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:16.651170 22481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:26:16.662165 22481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.1.keytab
--principal=oryx/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:43931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=36187
--tserver_master_addrs=127.8.131.62:45267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:16.663703 22481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:16.665081 22481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:16.679391 22520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:16.679446 22521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:16.680546 22524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:16.680912 22481 server_base.cc:1034] running on GCE node
I20260809 06:26:16.681589 22481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:16.682700 22481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:16.683934 22481 hybrid_clock.cc:648] HybridClock initialized: now 1786256776683860 us; error 56 us; skew 500 ppm
I20260809 06:26:16.684289 22481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:16 dist-test-slave-mjjr krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256776, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:16.688320 22481 init.cc:377] Logged in from keytab as oryx/127.8.131.1@KRBTEST.COM (short username oryx)
I20260809 06:26:16.690079 22481 webserver.cc:460] Webserver started at http://127.8.131.1:36187/ using document root <none> and password file <none>
I20260809 06:26:16.692930 22481 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:16.693100 22481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:16.701028 22481 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 06:26:16.704535 22533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:16.706283 22481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:26:16.706501 22481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "7a8e33ce303c4d8abee7b85d4c1ca4f0"
format_stamp: "Formatted at 2026-08-09 06:26:14 on dist-test-slave-mjjr"
server_key: "80b89220c952cbf0d330aa3b70de77b8"
server_key_iv: "e174e5559161bf8b933de361cff0627d"
server_key_version: "encryptionkey@0"
I20260809 06:26:16.708005 22481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:16.741369 22481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:16.746116 22481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:16.746416 22481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:16.747421 22481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:16.748800 22481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:16.748924 22481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:16.749056 22481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:16.749157 22481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:16.782718 22481 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:43931
I20260809 06:26:16.782814 22684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:43931 every 8 connection(s)
I20260809 06:26:16.784193 22481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:26:16.788079 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 22481
I20260809 06:26:16.789045 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:33515
--local_ip_for_outbound_sockets=127.8.131.2
--tserver_master_addrs=127.8.131.62:45267
--webserver_port=38139
--webserver_interface=127.8.131.2
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.131.2
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:26:16 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:16 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 06:26:16.810405 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:16.787108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:58861 (local address 127.8.131.62:45267)
0809 06:26:16.787846 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:16.787857 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:16.792684 (+ 4827us) server_negotiation.cc:407] Connection header received
0809 06:26:16.795407 (+ 2723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:16.795429 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:16.795930 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:16.796346 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:16.797779 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.799923 (+ 2144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.802175 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.802870 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.803017 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:16.809505 (+ 6488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:58861: BlockingRecv error: recv got EOF from 127.8.131.1:58861 (error 108)
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":287,"threads_started":1}
I20260809 06:26:16.810513 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:16.787096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:58861)
0809 06:26:16.792205 (+ 5109us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:16.792251 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:16.794702 (+ 2451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:16.796499 (+ 1797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:16.796507 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:16.797036 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:16.797569 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:16.797586 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.800135 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.800143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:16.802002 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:16.802022 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.803108 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.803118 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:16.803440 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:16.804230 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:16.804263 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:16.809188 (+ 4925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4818,"thread_start_us":4645,"threads_started":1}
W20260809 06:26:16.811301 22685 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:45267 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.131.62:45267: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Aug 09 06:26:16 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:16 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 06:26:16.820456 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:16.812148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:53647 (local address 127.8.131.62:45267)
0809 06:26:16.812364 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:16.812372 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:16.812394 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:26:16.812454 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:16.812462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:16.812548 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:16.812675 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:16.813372 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.814264 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.815548 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.816010 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.816146 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:16.820287 (+ 4141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:53647: BlockingRecv error: recv got EOF from 127.8.131.1:53647 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 06:26:16.820466 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:16.811997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:53647)
0809 06:26:16.812182 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:16.812203 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:16.812307 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:16.812717 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:16.812724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:16.813049 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:16.813186 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:16.813193 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.814415 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.814420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:16.815376 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:16.815394 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.816160 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.816166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:16.816347 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:16.816942 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:16.816959 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:16.820109 (+ 3150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
Aug 09 06:26:16 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:16 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:16.828986 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:16.821160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:45921)
0809 06:26:16.821340 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:16.821361 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:16.821460 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:16.821814 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:16.821819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:16.822145 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:16.822286 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:16.822294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.823373 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.823380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:16.824342 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:16.824353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.825042 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.825047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:16.825215 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:16.825789 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:16.825806 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:16.828686 (+ 2880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 06:26:16.829092 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:16.821267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:45921 (local address 127.8.131.62:45267)
0809 06:26:16.821437 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:16.821442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:16.821459 (+ 17us) server_negotiation.cc:407] Connection header received
0809 06:26:16.821604 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:16.821610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:16.821687 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:16.821789 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:16.822430 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.823232 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.824483 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:16.824896 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:16.825030 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:16.828878 (+ 3848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:45921: BlockingRecv error: recv got EOF from 127.8.131.1:45921 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 06:26:16.829331 22685 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 06:26:17.124491 22691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:17.124750 22691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:17.124843 22691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:17.132014 22691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:17.132131 22691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:17.132256 22691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:26:17.142071 22691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.2.keytab
--principal=oryx/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:33515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=38139
--tserver_master_addrs=127.8.131.62:45267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:17.143296 22691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:17.144596 22691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:17.157598 22702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:17.157648 22705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:17.157676 22699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:17.158773 22691 server_base.cc:1034] running on GCE node
I20260809 06:26:17.159307 22691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:17.160228 22691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:17.161418 22691 hybrid_clock.cc:648] HybridClock initialized: now 1786256777161365 us; error 49 us; skew 500 ppm
I20260809 06:26:17.161739 22691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256777, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:17.164943 22691 init.cc:377] Logged in from keytab as oryx/127.8.131.2@KRBTEST.COM (short username oryx)
I20260809 06:26:17.166402 22691 webserver.cc:460] Webserver started at http://127.8.131.2:38139/ using document root <none> and password file <none>
I20260809 06:26:17.167069 22691 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:17.167224 22691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:17.172508 22691 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:26:17.175010 22716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:17.176316 22691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:26:17.176502 22691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "5f952435737342e2949339049e9c0aff"
format_stamp: "Formatted at 2026-08-09 06:26:14 on dist-test-slave-mjjr"
server_key: "debdfbcf2234037109e2ee3d3c48ea60"
server_key_iv: "81bdaa9c65c595f6e8ae51bb2e475c64"
server_key_version: "encryptionkey@0"
I20260809 06:26:17.177438 22691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:17.189414 22691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:17.193058 22691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:17.193326 22691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:17.194275 22691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:17.195575 22691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:17.195732 22691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:17.195859 22691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:17.195953 22691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:17.229754 22691 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:33515
I20260809 06:26:17.229843 22869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:33515 every 8 connection(s)
I20260809 06:26:17.231312 22691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:26:17.240128 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 22691
I20260809 06:26:17.240723 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:40019
--local_ip_for_outbound_sockets=127.8.131.3
--tserver_master_addrs=127.8.131.62:45267
--webserver_port=34707
--webserver_interface=127.8.131.3
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.131.3
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 06:26:17.248361 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:17.233978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:43987 (local address 127.8.131.62:45267)
0809 06:26:17.234667 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:17.234671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:17.235625 (+ 954us) server_negotiation.cc:407] Connection header received
0809 06:26:17.237456 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:17.237463 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:17.237549 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:17.237669 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:17.239101 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.239913 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.241939 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.242380 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.242514 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.248170 (+ 5656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:43987: BlockingRecv error: recv got EOF from 127.8.131.2:43987 (error 108)
Metrics: {"server-negotiator.queue_time_us":568}
I20260809 06:26:17.249298 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:17.234660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:43987)
0809 06:26:17.235462 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:17.235512 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:17.237188 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:17.237893 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:17.237901 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:17.238392 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:17.238924 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.238941 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.240075 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.240079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.241798 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.241811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.242620 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.242628 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.242866 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.243557 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:17.243579 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:17.248003 (+ 4424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":315,"threads_started":1}
W20260809 06:26:17.249949 22870 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:45267 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.131.62:45267: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:17.258700 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:17.250599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:59969)
0809 06:26:17.250770 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:17.250790 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:17.250913 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:17.251246 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:17.251251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:17.251591 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:17.251806 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.251814 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.252847 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.252853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.253764 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.253775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.254497 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.254503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.254675 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.255301 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:17.255317 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:17.258406 (+ 3089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 06:26:17.259205 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:17.250750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:59969 (local address 127.8.131.62:45267)
0809 06:26:17.250953 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:17.250962 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:17.250995 (+ 33us) server_negotiation.cc:407] Connection header received
0809 06:26:17.251056 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:17.251060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:17.251117 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:17.251222 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:17.251949 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.252687 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.253918 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.254328 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.254476 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.258621 (+ 4145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:59969: BlockingRecv error: recv got EOF from 127.8.131.2:59969 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:17.267513 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:17.259415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:52011)
0809 06:26:17.259610 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:17.259626 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:17.259755 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:17.260193 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:17.260198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:17.260498 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:17.260634 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.260642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.261728 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.261733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.262620 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.262631 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.263335 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.263349 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.263520 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.264157 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:17.264173 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:17.267239 (+ 3066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 06:26:17.267988 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:17.259774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:52011 (local address 127.8.131.62:45267)
0809 06:26:17.259956 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:17.259960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:17.259974 (+ 14us) server_negotiation.cc:407] Connection header received
0809 06:26:17.260011 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:17.260015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:17.260064 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:17.260169 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:17.260773 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.261591 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.262766 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.263191 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.263327 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.267523 (+ 4196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:52011: BlockingRecv error: recv got EOF from 127.8.131.2:52011 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 06:26:17.271603 22870 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 06:26:17.577510 22873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:17.577824 22873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:17.578001 22873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:17.585342 22873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:17.585487 22873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:17.585669 22873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:26:17.595409 22873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/krb5kdc/oryx_127.8.131.3.keytab
--principal=oryx/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:40019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=34707
--tserver_master_addrs=127.8.131.62:45267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:17.596832 22873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:17.598119 22873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:17.611531 22881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:17.611493 22882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:17.611444 22884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:17.612389 22873 server_base.cc:1034] running on GCE node
I20260809 06:26:17.612959 22873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:17.614027 22873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:17.615244 22873 hybrid_clock.cc:648] HybridClock initialized: now 1786256777615196 us; error 43 us; skew 500 ppm
I20260809 06:26:17.615562 22873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256777, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:17.618690 22873 init.cc:377] Logged in from keytab as oryx/127.8.131.3@KRBTEST.COM (short username oryx)
I20260809 06:26:17.620133 22873 webserver.cc:460] Webserver started at http://127.8.131.3:34707/ using document root <none> and password file <none>
I20260809 06:26:17.620774 22873 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:17.620895 22873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:17.626122 22873 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 06:26:17.628834 22901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:17.630215 22873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:26:17.630369 22873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "66d214b8751f42888b63a314240df4c6"
format_stamp: "Formatted at 2026-08-09 06:26:15 on dist-test-slave-mjjr"
server_key: "312415be6f2ebe26208bf7cd75b7262c"
server_key_iv: "9e7d99736769ef9861802bd6d7b2e03a"
server_key_version: "encryptionkey@0"
I20260809 06:26:17.631414 22873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:17.652192 22873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:17.656394 22873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:17.656621 22873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:17.657506 22873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:17.658842 22873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:17.658931 22873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:17.659088 22873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:17.659162 22873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:17.694190 22873 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:40019
I20260809 06:26:17.694254 23056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:40019 every 8 connection(s)
I20260809 06:26:17.695783 22873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:26:17.698755 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 22873
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 06:26:17.712136 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:17.698252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:42173 (local address 127.8.131.62:45267)
0809 06:26:17.698441 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:17.698446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:17.700015 (+ 1569us) server_negotiation.cc:407] Connection header received
0809 06:26:17.702333 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:17.702338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:17.702428 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:17.702536 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:17.704017 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.704760 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.706493 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.706920 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.707052 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.711840 (+ 4788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:42173: BlockingRecv error: recv got EOF from 127.8.131.3:42173 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 06:26:17.712728 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:17.698917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:42173)
0809 06:26:17.699855 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:17.699906 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:17.702073 (+ 2167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:17.702771 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:17.702778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:17.703244 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:17.703850 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.703868 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.704911 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.704916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.706327 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.706338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.707161 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.707171 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.707392 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.708018 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:17.708045 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:17.711601 (+ 3556us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":334,"threads_started":1}
W20260809 06:26:17.713445 23057 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:45267 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.131.62:45267: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
I20260809 06:26:17.714521 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:17.699959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.8.131.62:45267)
0809 06:26:17.700858 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:17.700864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:17.700894 (+ 30us) server_negotiation.cc:407] Connection header received
0809 06:26:17.700961 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:17.700968 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:17.701050 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:17.701175 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:17.702661 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.703414 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.704771 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.705169 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.705306 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.708082 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:17.708090 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:17.708104 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:17.708137 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:17.711110 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:17.711807 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:17.711814 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:17.711824 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:17.711987 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:17.712439 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:17.712443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:17.712445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:17.712887 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:17.713192 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:17.713721 (+ 529us) server_negotiation.cc:299] Negotiation successful
0809 06:26:17.714198 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":666,"threads_started":1}
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:17.721695 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:17.714016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:39229)
0809 06:26:17.714246 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:17.714265 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:17.714410 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:17.714780 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:17.714787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:17.715181 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:17.715365 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.715377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.716433 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.716438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.717495 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.717506 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.718222 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.718228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.718395 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.718829 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:17.718845 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:17.721489 (+ 2644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127}
W20260809 06:26:17.721906 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:17.714074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:39229 (local address 127.8.131.62:45267)
0809 06:26:17.714223 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:17.714230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:17.714327 (+ 97us) server_negotiation.cc:407] Connection header received
0809 06:26:17.714529 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:17.714534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:17.714595 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:17.714751 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:17.715503 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.716288 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.717641 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.718072 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.718200 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.721685 (+ 3485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:39229: BlockingRecv error: recv got EOF from 127.8.131.3:39229 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:17.729844 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:17.722474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:47943)
0809 06:26:17.722674 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:17.722696 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:17.722794 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:17.723076 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:17.723081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:17.723367 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:17.723499 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.723506 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.724634 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.724639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.725513 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.725523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.726197 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.726203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.726368 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.726847 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:17.726863 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:17.729504 (+ 2641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260809 06:26:17.729923 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:17.722548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:47943 (local address 127.8.131.62:45267)
0809 06:26:17.722817 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:17.722825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:17.722844 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:26:17.722890 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:17.722895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:17.722952 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:17.723051 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:17.723632 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.724485 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.725675 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.726049 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.726181 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.729676 (+ 3495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:47943: BlockingRecv error: recv got EOF from 127.8.131.3:47943 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
W20260809 06:26:17.730284 23057 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:17 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:17.839133 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:17.830141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:49605)
0809 06:26:17.830736 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:17.830759 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:17.830876 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:17.831190 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:17.831193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:17.831765 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:17.831965 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.831975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.832942 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.832945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.833927 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:17.833938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.834861 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.834868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:17.834999 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.835624 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:17.835662 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:17.838696 (+ 3034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":341,"threads_started":1}
W20260809 06:26:17.839217 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:17.830096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:49605 (local address 127.8.131.62:45267)
0809 06:26:17.830295 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:17.830299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:17.830836 (+ 537us) server_negotiation.cc:407] Connection header received
0809 06:26:17.830975 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:17.830979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:17.831038 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:17.831132 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:17.832100 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.832814 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.834075 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:17.834598 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:17.834721 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:17.838939 (+ 4218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:49605: BlockingRecv error: recv got EOF from 127.8.131.1:49605 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 06:26:18 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:18 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:18.280440 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:18.272255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:39079)
0809 06:26:18.272730 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:18.272750 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:18.272858 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:18.273128 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:18.273131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:18.273622 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:18.273813 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:18.273822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.274773 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.274776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:18.275752 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:18.275763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.276533 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.276539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:18.276720 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:18.277249 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:18.277265 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:18.280078 (+ 2813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":264,"threads_started":1}
W20260809 06:26:18.280599 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:18.272368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:39079 (local address 127.8.131.62:45267)
0809 06:26:18.272586 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:18.272593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:18.272830 (+ 237us) server_negotiation.cc:407] Connection header received
0809 06:26:18.272939 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:18.272943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:18.273000 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:18.273076 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:18.273933 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.274650 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.275910 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.276344 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.276491 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:18.280343 (+ 3852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:39079: BlockingRecv error: recv got EOF from 127.8.131.2:39079 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 06:26:18 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:18 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:18.744568 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:18.730992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:49099)
0809 06:26:18.731802 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:18.731818 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:18.731948 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:18.732269 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:18.732273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:18.732768 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:18.733005 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:18.733014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.734120 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.734123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:18.735238 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:18.735250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.736317 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.736326 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:18.736510 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:18.741259 (+ 4749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:18.741285 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:18.744327 (+ 3042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":529,"threads_started":1}
W20260809 06:26:18.744815 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:18.731077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:49099 (local address 127.8.131.62:45267)
0809 06:26:18.731350 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:18.731359 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:18.731915 (+ 556us) server_negotiation.cc:407] Connection header received
0809 06:26:18.732035 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:18.732039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:18.732101 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:18.732191 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:18.733136 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.733947 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.735416 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.736079 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.736224 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:18.744566 (+ 8342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:49099: BlockingRecv error: recv got EOF from 127.8.131.3:49099 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 06:26:18 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:18 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:18.848546 23069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:18.840184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:56253)
0809 06:26:18.840777 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:18.840798 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:18.840912 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:18.841225 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:18.841228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:18.841660 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:18.841815 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:18.841825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.842849 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.842853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:18.844024 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:18.844041 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.844840 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.844844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:18.844954 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:18.845522 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:18.845539 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:18.848302 (+ 2763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":337,"threads_started":1}
W20260809 06:26:18.848742 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:18.840277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:56253 (local address 127.8.131.62:45267)
0809 06:26:18.840509 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:18.840514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:18.840872 (+ 358us) server_negotiation.cc:407] Connection header received
0809 06:26:18.841004 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:18.841008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:18.841085 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:18.841168 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:18.841936 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.842705 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.844156 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:18.844674 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:18.844791 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:18.848493 (+ 3702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:56253: BlockingRecv error: recv got EOF from 127.8.131.1:56253 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 06:26:19 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:19 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:19.289716 23070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:19.281440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:47301)
0809 06:26:19.281931 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:19.281952 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:19.282121 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:19.282436 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:19.282440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:19.282857 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:19.283010 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:19.283018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.284016 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.284020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:19.284913 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:19.284925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.285548 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.285552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:19.285657 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:19.286222 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:19.286239 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:19.289359 (+ 3120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":261,"threads_started":1}
W20260809 06:26:19.289827 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:19.281494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:47301 (local address 127.8.131.62:45267)
0809 06:26:19.281733 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:19.281740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:19.282051 (+ 311us) server_negotiation.cc:407] Connection header received
0809 06:26:19.282244 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:19.282248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:19.282309 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:19.282395 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:19.283139 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.283885 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.285029 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.285427 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.285514 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:19.289568 (+ 4054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:47301: BlockingRecv error: recv got EOF from 127.8.131.2:47301 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 06:26:19 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:19 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:19.754645 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:19.745535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:53451)
0809 06:26:19.746123 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:19.746138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:19.746297 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:19.746656 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:19.746660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:19.747112 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:19.747256 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:19.747265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.748405 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.748410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:19.749601 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:19.749613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.750687 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.750692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:19.750836 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:19.751426 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:19.751445 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:19.754304 (+ 2859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":318,"threads_started":1}
W20260809 06:26:19.754994 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:19.745737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:53451 (local address 127.8.131.62:45267)
0809 06:26:19.746002 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:19.746009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:19.746218 (+ 209us) server_negotiation.cc:407] Connection header received
0809 06:26:19.746424 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:19.746430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:19.746519 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:19.746619 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:19.747410 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.748242 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.749740 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.750509 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.750663 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:19.754607 (+ 3944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:53451: BlockingRecv error: recv got EOF from 127.8.131.3:53451 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 06:26:19 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:19 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:19.858517 23072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:19.849521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:53055)
0809 06:26:19.850127 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:19.850144 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:19.850311 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:19.850678 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:19.850683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:19.851084 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:19.851269 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:19.851278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.852402 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.852410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:19.853351 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:19.853362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.854048 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.854054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:19.854238 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:19.854880 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:19.854898 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:19.858264 (+ 3366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":342,"threads_started":1}
W20260809 06:26:19.858726 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:19.849689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:53055 (local address 127.8.131.62:45267)
0809 06:26:19.849959 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:19.849967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:19.850241 (+ 274us) server_negotiation.cc:407] Connection header received
0809 06:26:19.850450 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:19.850457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:19.850530 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:19.850648 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:19.851425 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.852243 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.853502 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:19.853892 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:19.854023 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:19.858466 (+ 4443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:53055: BlockingRecv error: recv got EOF from 127.8.131.1:53055 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 09 06:26:20 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:20 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:20.299957 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:20.290851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:60127)
0809 06:26:20.291393 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:20.291421 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:20.291584 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:20.292550 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:20.292555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:20.293058 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:20.293231 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:20.293240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.294397 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.294402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:20.295277 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:20.295289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.296094 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.296097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:20.296216 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:20.296813 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:20.296829 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:20.299543 (+ 2714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":287,"threads_started":1}
W20260809 06:26:20.300139 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:20.291260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:60127 (local address 127.8.131.62:45267)
0809 06:26:20.291465 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:20.291470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:20.291489 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:26:20.292242 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:20.292248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:20.292358 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:20.292480 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:20.293392 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.294250 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.295407 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.295966 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.296078 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:20.299847 (+ 3769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:60127: BlockingRecv error: recv got EOF from 127.8.131.2:60127 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 06:26:20 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:20 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:20.765738 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:20.755807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:33823)
0809 06:26:20.756319 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:20.756335 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:20.756518 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:20.756885 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:20.756890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:20.757280 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:20.757498 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:20.757513 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.759213 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.759219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:20.760695 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:20.760710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.761352 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.761356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:20.761504 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:20.762082 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:20.762100 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:20.765482 (+ 3382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":276,"threads_started":1}
W20260809 06:26:20.765900 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:20.755852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:33823 (local address 127.8.131.62:45267)
0809 06:26:20.756071 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:20.756078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:20.756419 (+ 341us) server_negotiation.cc:407] Connection header received
0809 06:26:20.756645 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:20.756651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:20.756741 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:20.756851 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:20.757729 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.759030 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.760827 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.761219 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.761315 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:20.765673 (+ 4358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:33823: BlockingRecv error: recv got EOF from 127.8.131.3:33823 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 06:26:20 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:20 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:20.868023 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:20.859576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:43131)
0809 06:26:20.860172 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:20.860199 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:20.860364 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:20.860734 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:20.860738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:20.861139 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:20.861284 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:20.861292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.862337 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.862344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:20.863297 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:20.863308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.864115 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.864120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:20.864321 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:20.864900 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:20.864917 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:20.867682 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":328,"threads_started":1}
W20260809 06:26:20.868170 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:20.859711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:43131 (local address 127.8.131.62:45267)
0809 06:26:20.859933 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:20.859942 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:20.860295 (+ 353us) server_negotiation.cc:407] Connection header received
0809 06:26:20.860495 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:20.860506 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:20.860604 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:20.860700 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:20.861432 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.862193 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.863453 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:20.863953 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:20.864092 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:20.867918 (+ 3826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:43131: BlockingRecv error: recv got EOF from 127.8.131.1:43131 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 06:26:21 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:21 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:21.311018 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:21.301170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:46135)
0809 06:26:21.301821 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:21.301851 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:21.302027 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:21.302474 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:21.302479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:21.302974 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:21.303135 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:21.303144 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.304303 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.304311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:21.305458 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:21.305472 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.306254 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.306260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:21.306488 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:21.307180 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:21.307199 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:21.310647 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":347,"threads_started":1}
W20260809 06:26:21.311120 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:21.301238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:46135 (local address 127.8.131.62:45267)
0809 06:26:21.301515 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:21.301525 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:21.301958 (+ 433us) server_negotiation.cc:407] Connection header received
0809 06:26:21.302188 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:21.302194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:21.302313 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:21.302437 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:21.303311 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.304164 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.305625 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.306087 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.306241 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:21.310862 (+ 4621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:46135: BlockingRecv error: recv got EOF from 127.8.131.2:46135 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 09 06:26:21 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:21 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:21.775233 23082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:21.766721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:54049)
0809 06:26:21.767351 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:21.767368 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:21.767492 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:21.767856 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:21.767860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:21.768234 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:21.768392 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:21.768401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.769468 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.769471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:21.770427 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:21.770437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.771121 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.771124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:21.771250 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:21.772015 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:21.772033 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:21.774997 (+ 2964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":273,"threads_started":1}
W20260809 06:26:21.775418 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:21.766777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:54049 (local address 127.8.131.62:45267)
0809 06:26:21.766969 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:21.766979 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:21.767456 (+ 477us) server_negotiation.cc:407] Connection header received
0809 06:26:21.767631 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:21.767636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:21.767716 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:21.767815 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:21.768555 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.769354 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.770548 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.770998 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.771092 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:21.775189 (+ 4097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:54049: BlockingRecv error: recv got EOF from 127.8.131.3:54049 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 06:26:21 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:21 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:21.883608 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:21.874810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:56001)
0809 06:26:21.875626 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:21.875838 (+ 212us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:21.875988 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:21.876516 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:21.876521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:21.877023 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:21.877187 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:21.877196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.878178 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.878181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:21.879088 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:21.879099 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.879831 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.879835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:21.879956 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:21.880488 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:21.880504 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:21.883323 (+ 2819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":521,"threads_started":1}
W20260809 06:26:21.883872 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:21.875124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:56001 (local address 127.8.131.62:45267)
0809 06:26:21.875321 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:21.875327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:21.875929 (+ 602us) server_negotiation.cc:407] Connection header received
0809 06:26:21.876172 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:21.876178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:21.876268 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:21.876372 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:21.877335 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.878050 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.879220 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:21.879696 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:21.879830 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:21.883571 (+ 3741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:56001: BlockingRecv error: recv got EOF from 127.8.131.1:56001 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 06:26:22 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:22 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:22.321290 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:22.312211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:41369)
0809 06:26:22.312768 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:22.312788 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:22.312895 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:22.313250 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:22.313253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:22.313666 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:22.313825 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:22.313833 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.314991 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.314994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:22.315940 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:22.315952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.316688 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.316692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:22.316885 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:22.317608 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:22.317637 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:22.320872 (+ 3235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":293,"threads_started":1}
W20260809 06:26:22.321393 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:22.312374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:41369 (local address 127.8.131.62:45267)
0809 06:26:22.312599 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:22.312603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:22.312866 (+ 263us) server_negotiation.cc:407] Connection header received
0809 06:26:22.312997 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:22.313002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:22.313103 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:22.313229 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:22.313972 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.314860 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.316067 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.316524 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.316689 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:22.321092 (+ 4403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:41369: BlockingRecv error: recv got EOF from 127.8.131.2:41369 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 06:26:22 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:22 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:22.784999 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:22.776258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:51563)
0809 06:26:22.776800 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:22.776816 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:22.776931 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:22.777221 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:22.777224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:22.777624 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:22.777791 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:22.777798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.778770 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.778773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:22.779669 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:22.779679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.780524 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.780528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:22.780643 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:22.781283 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:22.781301 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:22.784637 (+ 3336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":271,"threads_started":1}
W20260809 06:26:22.785084 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:22.776358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:51563 (local address 127.8.131.62:45267)
0809 06:26:22.776617 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:22.776623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:22.776898 (+ 275us) server_negotiation.cc:407] Connection header received
0809 06:26:22.777031 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:22.777035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:22.777096 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:22.777178 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:22.777911 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.778638 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.779832 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.780363 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.780503 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:22.784835 (+ 4332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:51563: BlockingRecv error: recv got EOF from 127.8.131.3:51563 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 06:26:22 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:22 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:22.893095 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:22.884552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:57695)
0809 06:26:22.885108 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:22.885124 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:22.885239 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:22.885599 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:22.885603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:22.885991 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:22.886140 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:22.886149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.887198 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.887201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:22.888176 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:22.888187 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.888828 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.888831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:22.888946 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:22.889565 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:22.889581 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:22.892727 (+ 3146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":322,"threads_started":1}
W20260809 06:26:22.893383 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:22.884621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:57695 (local address 127.8.131.62:45267)
0809 06:26:22.884816 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:22.884821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:22.885220 (+ 399us) server_negotiation.cc:407] Connection header received
0809 06:26:22.885362 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:22.885369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:22.885454 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:22.885564 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:22.886276 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.887073 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.888316 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:22.888697 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:22.888791 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:22.893098 (+ 4307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:57695: BlockingRecv error: recv got EOF from 127.8.131.1:57695 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 06:26:23 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:23 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:23.331120 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:23.322336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:39157)
0809 06:26:23.322882 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:23.322903 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:23.323065 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:23.323425 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:23.323430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:23.324028 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:23.324198 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:23.324208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.325227 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.325234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:23.326231 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:23.326242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.326985 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.326991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:23.327164 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:23.327789 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:23.327805 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:23.330806 (+ 3001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":291,"threads_started":1}
W20260809 06:26:23.331486 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:23.322422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:39157 (local address 127.8.131.62:45267)
0809 06:26:23.322683 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:23.322692 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:23.322992 (+ 300us) server_negotiation.cc:407] Connection header received
0809 06:26:23.323188 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:23.323194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:23.323266 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:23.323389 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:23.324364 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.325092 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.326399 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.326825 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.326970 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:23.331228 (+ 4258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:39157: BlockingRecv error: recv got EOF from 127.8.131.2:39157 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 06:26:23 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:23 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:23.798940 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:23.785996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:51059)
0809 06:26:23.786580 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:23.786595 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:23.786755 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:23.787094 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:23.787099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:23.787506 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:23.787728 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:23.787738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.788785 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.788790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:23.789683 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:23.789694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.790532 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.790538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:23.790711 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:23.795907 (+ 5196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:23.795926 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:23.798708 (+ 2782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":319,"threads_started":1}
W20260809 06:26:23.799237 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:23.786067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:51059 (local address 127.8.131.62:45267)
0809 06:26:23.786321 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:23.786328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:23.786682 (+ 354us) server_negotiation.cc:407] Connection header received
0809 06:26:23.786896 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:23.786900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:23.786959 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:23.787060 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:23.787865 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.788644 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.789837 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.790365 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.790517 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:23.798951 (+ 8434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:51059: BlockingRecv error: recv got EOF from 127.8.131.3:51059 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 06:26:23 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:23 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:23.902966 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:23.894457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:56495)
0809 06:26:23.895065 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:23.895081 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:23.895238 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:23.895566 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:23.895571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:23.896023 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:23.896187 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:23.896195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.897263 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.897268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:23.898183 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:23.898193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.898945 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.898950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:23.899144 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:23.899731 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:23.899747 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:23.902550 (+ 2803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":266,"threads_started":1}
W20260809 06:26:23.903007 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:23.894573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:56495 (local address 127.8.131.62:45267)
0809 06:26:23.894853 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:23.894860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:23.895166 (+ 306us) server_negotiation.cc:407] Connection header received
0809 06:26:23.895359 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:23.895364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:23.895433 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:23.895533 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:23.896348 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.897115 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.898337 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:23.898786 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:23.898933 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:23.902753 (+ 3820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:56495: BlockingRecv error: recv got EOF from 127.8.131.1:56495 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 06:26:24 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:24 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:24.341456 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:24.332180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:53493)
0809 06:26:24.332895 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:24.332916 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:24.333043 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:24.333354 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:24.333357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:24.333825 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:24.333997 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:24.334006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.335140 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.335146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:24.336183 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:24.336196 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.336914 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.336920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:24.337098 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:24.337690 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:24.337717 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:24.341121 (+ 3404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":358,"threads_started":1}
W20260809 06:26:24.341583 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:24.332253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:53493 (local address 127.8.131.62:45267)
0809 06:26:24.332430 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:24.332434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:24.332993 (+ 559us) server_negotiation.cc:407] Connection header received
0809 06:26:24.333142 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:24.333146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:24.333226 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:24.333310 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:24.334121 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.334970 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.336345 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.336741 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.336901 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:24.341351 (+ 4450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:53493: BlockingRecv error: recv got EOF from 127.8.131.2:53493 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 06:26:24 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:24 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:24.808971 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:24.800104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:34053)
0809 06:26:24.800658 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:24.800673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:24.800788 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:24.801083 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:24.801086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:24.801514 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:24.801680 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:24.801688 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.802648 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.802651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:24.803591 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:24.803602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.804490 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.804496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:24.804653 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:24.805484 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:24.805508 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:24.808707 (+ 3199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":247,"threads_started":1}
W20260809 06:26:24.809101 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:24.800157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:34053 (local address 127.8.131.62:45267)
0809 06:26:24.800406 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:24.800416 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:24.800758 (+ 342us) server_negotiation.cc:407] Connection header received
0809 06:26:24.800894 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:24.800898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:24.800957 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:24.801036 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:24.801803 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.802523 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.803737 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.804331 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.804434 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:24.808905 (+ 4471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:34053: BlockingRecv error: recv got EOF from 127.8.131.3:34053 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 06:26:24 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:24 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:24.912423 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:24.903977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:52865)
0809 06:26:24.904561 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:24.904577 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:24.904744 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:24.905090 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:24.905094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:24.905541 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:24.905689 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:24.905699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.906732 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.906739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:24.907668 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:24.907679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.908467 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.908473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:24.908640 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:24.909206 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:24.909223 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:24.912140 (+ 2917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":308,"threads_started":1}
W20260809 06:26:24.912667 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:24.904146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:52865 (local address 127.8.131.62:45267)
0809 06:26:24.904393 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:24.904399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:24.904672 (+ 273us) server_negotiation.cc:407] Connection header received
0809 06:26:24.904871 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:24.904883 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:24.904944 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:24.905055 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:24.905839 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.906589 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.907809 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:24.908274 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:24.908414 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:24.912376 (+ 3962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:52865: BlockingRecv error: recv got EOF from 127.8.131.1:52865 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 06:26:25 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:25 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:25.351019 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:25.342618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:54999)
0809 06:26:25.343264 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:25.343284 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:25.343405 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:25.343719 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:25.343722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:25.344138 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:25.344297 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:25.344305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.345467 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.345471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:25.346355 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:25.346366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.347080 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.347085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:25.347211 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:25.347804 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:25.347822 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:25.350672 (+ 2850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":375,"threads_started":1}
W20260809 06:26:25.351130 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:25.342716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:54999 (local address 127.8.131.62:45267)
0809 06:26:25.342922 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:25.342930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:25.343362 (+ 432us) server_negotiation.cc:407] Connection header received
0809 06:26:25.343503 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:25.343507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:25.343568 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:25.343664 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:25.344422 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.345333 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.346503 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.346929 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.347022 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:25.350868 (+ 3846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:54999: BlockingRecv error: recv got EOF from 127.8.131.2:54999 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 06:26:25 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:25 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:25.819473 23100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:25.810109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:35291)
0809 06:26:25.810666 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:25.810682 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:25.810848 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:25.811205 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:25.811209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:25.811687 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:25.811840 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:25.811849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.812891 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.812896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:25.813915 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:25.813927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.814771 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.814778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:25.814997 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:25.815771 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:25.815795 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:25.819123 (+ 3328us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":279,"threads_started":1}
W20260809 06:26:25.819574 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:25.810176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:35291 (local address 127.8.131.62:45267)
0809 06:26:25.810439 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:25.810447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:25.810776 (+ 329us) server_negotiation.cc:407] Connection header received
0809 06:26:25.810982 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:25.810987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:25.811059 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:25.811162 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:25.811987 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.812727 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.814051 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.814484 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.814656 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:25.819327 (+ 4671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:35291: BlockingRecv error: recv got EOF from 127.8.131.3:35291 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 06:26:25 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:25 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:25.921892 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:25.913466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:54955)
0809 06:26:25.914132 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:25.914149 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:25.914260 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:25.914557 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:25.914560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:25.914964 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:25.915159 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:25.915168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.916366 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.916369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:25.917335 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:25.917346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.917981 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.917985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:25.918102 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:25.918651 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:25.918667 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:25.921588 (+ 2921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":441,"threads_started":1}
W20260809 06:26:25.922124 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:25.913484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:54955 (local address 127.8.131.62:45267)
0809 06:26:25.913713 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:25.913722 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:25.914227 (+ 505us) server_negotiation.cc:407] Connection header received
0809 06:26:25.914363 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:25.914367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:25.914434 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:25.914514 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:25.915299 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.916255 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.917457 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:25.917855 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:25.917947 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:25.921850 (+ 3903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:54955: BlockingRecv error: recv got EOF from 127.8.131.1:54955 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 06:26:26 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:26 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:26.360782 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:26.351981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:37091)
0809 06:26:26.352589 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:26.352611 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:26.352726 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:26.353030 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:26.353033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:26.353457 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:26.353627 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:26.353636 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.354675 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.354678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:26.355702 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:26.355721 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.356384 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.356388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:26.356510 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:26.357084 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:26.357101 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:26.360406 (+ 3305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":330,"threads_started":1}
W20260809 06:26:26.360877 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:26.352163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:37091 (local address 127.8.131.62:45267)
0809 06:26:26.352327 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:26.352331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:26.352685 (+ 354us) server_negotiation.cc:407] Connection header received
0809 06:26:26.352824 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:26.352828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:26.352896 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:26.352979 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:26.353746 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.354534 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.355836 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.356239 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.356327 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:26.360605 (+ 4278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:37091: BlockingRecv error: recv got EOF from 127.8.131.2:37091 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 06:26:26 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:26 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:26.829559 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:26.820586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:56511)
0809 06:26:26.821154 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:26.821170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:26.821334 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:26.821686 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:26.821692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:26.822137 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:26.822316 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:26.822325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.823397 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.823404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:26.824632 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:26.824645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.825530 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.825536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:26.825720 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:26.826288 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:26.826306 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:26.829207 (+ 2901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":361,"threads_started":1}
W20260809 06:26:26.829653 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:26.820700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:56511 (local address 127.8.131.62:45267)
0809 06:26:26.820936 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:26.820942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:26.821263 (+ 321us) server_negotiation.cc:407] Connection header received
0809 06:26:26.821469 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:26.821478 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:26.821538 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:26.821641 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:26.822472 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.823246 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.824775 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.825380 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.825513 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:26.829416 (+ 3903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:56511: BlockingRecv error: recv got EOF from 127.8.131.3:56511 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 06:26:26 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:26 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:26.930990 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:26.922860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:60963)
0809 06:26:26.923420 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:26.923438 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:26.923584 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:26.923909 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:26.923912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:26.924329 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:26.924478 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:26.924499 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.925540 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.925544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:26.926487 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:26.926500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.927197 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.927202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:26.927335 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:26.927909 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:26.927928 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:26.930767 (+ 2839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":343,"threads_started":1}
W20260809 06:26:26.931190 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:26.922982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:60963 (local address 127.8.131.62:45267)
0809 06:26:26.923327 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:26.923333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:26.923547 (+ 214us) server_negotiation.cc:407] Connection header received
0809 06:26:26.923698 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:26.923702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:26.923762 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:26.923856 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:26.924617 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.925399 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.926613 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:26.927059 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:26.927148 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:26.930926 (+ 3778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:60963: BlockingRecv error: recv got EOF from 127.8.131.1:60963 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 09 06:26:27 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:27 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:27.370797 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:27.361875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:46641)
0809 06:26:27.362502 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:27.362529 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:27.362682 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:27.363021 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:27.363024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:27.363421 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:27.363598 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:27.363607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.364676 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.364679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:27.365690 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:27.365702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.366411 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.366415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:27.366571 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:27.367200 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:27.367219 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:27.370453 (+ 3234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":374,"threads_started":1}
W20260809 06:26:27.370913 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:27.361930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:46641 (local address 127.8.131.62:45267)
0809 06:26:27.362110 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:27.362114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:27.362609 (+ 495us) server_negotiation.cc:407] Connection header received
0809 06:26:27.362823 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:27.362827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:27.362887 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:27.362968 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:27.363828 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.364544 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.365829 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.366267 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.366357 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:27.370660 (+ 4303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:46641: BlockingRecv error: recv got EOF from 127.8.131.2:46641 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 06:26:27 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:27 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:27.839462 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:27.830732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:45333)
0809 06:26:27.831335 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:27.831356 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:27.831477 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:27.831763 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:27.831767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:27.832178 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:27.832325 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:27.832334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.833382 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.833386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:27.834610 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:27.834627 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.835586 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.835592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:27.835800 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:27.836384 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:27.836401 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:27.839230 (+ 2829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":336,"threads_started":1}
W20260809 06:26:27.839757 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:27.830857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:45333 (local address 127.8.131.62:45267)
0809 06:26:27.831037 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:27.831042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:27.831435 (+ 393us) server_negotiation.cc:407] Connection header received
0809 06:26:27.831571 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:27.831575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:27.831633 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:27.831731 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:27.832478 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.833255 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.834762 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.835451 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.835573 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:27.839482 (+ 3909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:45333: BlockingRecv error: recv got EOF from 127.8.131.3:45333 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 06:26:27 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:27 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:27.940784 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:27.932037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:44843)
0809 06:26:27.932603 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:27.932620 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:27.932734 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:27.933009 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:27.933012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:27.933421 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:27.933611 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:27.933621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.934700 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.934705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:27.935893 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:27.935908 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.936709 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.936713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:27.936849 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:27.937428 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:27.937446 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:27.940527 (+ 3081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":311,"threads_started":1}
W20260809 06:26:27.940938 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:27.932092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:44843 (local address 127.8.131.62:45267)
0809 06:26:27.932270 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:27.932274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:27.932690 (+ 416us) server_negotiation.cc:407] Connection header received
0809 06:26:27.932837 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:27.932841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:27.932915 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:27.932996 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:27.933736 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.934547 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.936046 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:27.936574 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:27.936671 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:27.940723 (+ 4052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:44843: BlockingRecv error: recv got EOF from 127.8.131.1:44843 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 06:26:28 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:28 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:28.380362 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:28.371865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:46583)
0809 06:26:28.372349 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:28.372370 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:28.372491 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:28.372774 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:28.372778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:28.373169 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:28.373343 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:28.373350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.374468 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.374471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:28.375565 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:28.375578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.376675 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.376679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:28.376793 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:28.377368 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:28.377395 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:28.380119 (+ 2724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":281,"threads_started":1}
W20260809 06:26:28.380605 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:28.371990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:46583 (local address 127.8.131.62:45267)
0809 06:26:28.372246 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:28.372253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:28.372461 (+ 208us) server_negotiation.cc:407] Connection header received
0809 06:26:28.372590 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:28.372594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:28.372656 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:28.372733 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:28.373481 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.374349 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.375758 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.376526 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.376641 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:28.380314 (+ 3673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:46583: BlockingRecv error: recv got EOF from 127.8.131.2:46583 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 09 06:26:28 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:28 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:28.853070 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:28.840424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:45243)
0809 06:26:28.840987 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:28.841017 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:28.841173 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:28.841539 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:28.841544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:28.841996 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:28.842191 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:28.842201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.843285 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.843294 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:28.844298 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:28.844310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.845008 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.845014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:28.845182 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:28.849816 (+ 4634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:28.849835 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:28.852780 (+ 2945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":283,"threads_started":1}
W20260809 06:26:28.853253 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:28.840522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:45243 (local address 127.8.131.62:45267)
0809 06:26:28.840755 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:28.840762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:28.841099 (+ 337us) server_negotiation.cc:407] Connection header received
0809 06:26:28.841322 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:28.841328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:28.841402 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:28.841503 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:28.842351 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.843149 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.844446 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.844853 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.844992 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:28.852991 (+ 7999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:45243: BlockingRecv error: recv got EOF from 127.8.131.3:45243 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 06:26:28 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:28 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:28.950170 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:28.941819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:49867)
0809 06:26:28.942451 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:28.942473 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:28.942621 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:28.943054 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:28.943064 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:28.943621 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:28.943810 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:28.943820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.944779 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.944782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:28.945680 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:28.945690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.946438 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.946442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:28.946577 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:28.947218 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:28.947236 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:28.949939 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":350,"threads_started":1}
W20260809 06:26:28.950397 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:28.941876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:49867 (local address 127.8.131.62:45267)
0809 06:26:28.942081 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:28.942089 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:28.942562 (+ 473us) server_negotiation.cc:407] Connection header received
0809 06:26:28.942758 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:28.942766 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:28.942881 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:28.942961 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:28.943938 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.944657 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.945829 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:28.946296 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:28.946397 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:28.950139 (+ 3742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:49867: BlockingRecv error: recv got EOF from 127.8.131.1:49867 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 06:26:29 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:29 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:29.389642 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:29.381226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:42991)
0809 06:26:29.381881 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:29.381904 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:29.382073 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:29.382493 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:29.382499 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:29.383032 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:29.383244 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:29.383253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.384297 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.384301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:29.385206 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:29.385235 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.385933 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.385937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:29.386063 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:29.386655 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:29.386671 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:29.389379 (+ 2708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":395,"threads_started":1}
W20260809 06:26:29.389812 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:29.381302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:42991 (local address 127.8.131.62:45267)
0809 06:26:29.381548 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:29.381557 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:29.382009 (+ 452us) server_negotiation.cc:407] Connection header received
0809 06:26:29.382221 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:29.382227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:29.382320 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:29.382410 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:29.383406 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.384168 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.385355 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.385811 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.385912 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:29.389594 (+ 3682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:42991: BlockingRecv error: recv got EOF from 127.8.131.2:42991 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 06:26:29 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:29 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:29.862447 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:29.854078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:49539)
0809 06:26:29.854588 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:29.854605 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:29.854716 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:29.855041 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:29.855044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:29.855424 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:29.855579 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:29.855586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.856635 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.856639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:29.857563 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:29.857573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.858237 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.858241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:29.858364 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:29.858971 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:29.858988 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:29.862058 (+ 3070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":287,"threads_started":1}
W20260809 06:26:29.862521 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:29.854115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:49539 (local address 127.8.131.62:45267)
0809 06:26:29.854366 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:29.854376 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:29.854689 (+ 313us) server_negotiation.cc:407] Connection header received
0809 06:26:29.854821 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:29.854826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:29.854905 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:29.854989 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:29.855764 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.856510 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.857686 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.858116 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.858210 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:29.862260 (+ 4050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:49539: BlockingRecv error: recv got EOF from 127.8.131.3:49539 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 06:26:29 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:29 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:29.959843 23123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:29.951198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:51569)
0809 06:26:29.951793 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:29.951810 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:29.951923 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:29.952209 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:29.952212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:29.952595 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:29.952745 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:29.952753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.953790 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.953794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:29.954725 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:29.954737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.955724 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.955731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:29.955900 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:29.956596 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:29.956620 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:29.959466 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":290,"threads_started":1}
W20260809 06:26:29.959962 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:29.951298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:51569 (local address 127.8.131.62:45267)
0809 06:26:29.951576 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:29.951582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:29.951892 (+ 310us) server_negotiation.cc:407] Connection header received
0809 06:26:29.952008 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:29.952011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:29.952085 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:29.952165 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:29.952887 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.953655 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.954873 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:29.955484 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:29.955601 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:29.959693 (+ 4092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:51569: BlockingRecv error: recv got EOF from 127.8.131.1:51569 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 06:26:30 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:30 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:30.399369 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:30.390541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:53119)
0809 06:26:30.391128 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:30.391146 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:30.391298 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:30.391677 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:30.391682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:30.392105 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:30.392269 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:30.392278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.393319 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.393325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:30.394280 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:30.394292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.395046 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.395052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:30.395226 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:30.395923 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:30.395940 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:30.399037 (+ 3097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":329,"threads_started":1}
W20260809 06:26:30.399506 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:30.390748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:53119 (local address 127.8.131.62:45267)
0809 06:26:30.390975 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:30.390982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:30.391228 (+ 246us) server_negotiation.cc:407] Connection header received
0809 06:26:30.391445 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:30.391450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:30.391513 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:30.391611 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:30.392413 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.393173 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.394435 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.394861 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.395006 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:30.399236 (+ 4230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:53119: BlockingRecv error: recv got EOF from 127.8.131.2:53119 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 06:26:30 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:30 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:30.872076 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:30.863504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:45913)
0809 06:26:30.864086 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:30.864102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:30.864259 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:30.864628 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:30.864634 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:30.865074 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:30.865232 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:30.865241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.866263 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.866269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:30.867224 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:30.867235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.868056 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.868061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:30.868223 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:30.868864 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:30.868888 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:30.871835 (+ 2947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":325,"threads_started":1}
W20260809 06:26:30.872211 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:30.863563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:45913 (local address 127.8.131.62:45267)
0809 06:26:30.863837 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:30.863845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:30.864193 (+ 348us) server_negotiation.cc:407] Connection header received
0809 06:26:30.864397 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:30.864408 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:30.864487 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:30.864588 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:30.865391 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.866120 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.867379 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.867863 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.867993 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:30.872012 (+ 4019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:45913: BlockingRecv error: recv got EOF from 127.8.131.3:45913 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 09 06:26:30 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:30 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:30.969828 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:30.960809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:34231)
0809 06:26:30.961309 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:30.961327 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:30.961441 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:30.961801 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:30.961804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:30.962186 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:30.962374 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:30.962382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.963478 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.963481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:30.964410 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:30.964421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.965145 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.965149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:30.965269 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:30.966079 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:30.966102 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:30.969543 (+ 3441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":294,"threads_started":1}
W20260809 06:26:30.969975 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:30.960882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:34231 (local address 127.8.131.62:45267)
0809 06:26:30.961251 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:30.961259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:30.961412 (+ 153us) server_negotiation.cc:407] Connection header received
0809 06:26:30.961548 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:30.961552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:30.961660 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:30.961768 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:30.962554 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.963349 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.964551 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:30.965017 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:30.965116 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:30.969735 (+ 4619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:34231: BlockingRecv error: recv got EOF from 127.8.131.1:34231 (error 108)
Metrics: {"server-negotiator.queue_time_us":200}
Aug 09 06:26:31 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:31 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:31.408641 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:31.400500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:34703)
0809 06:26:31.400947 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:31.400968 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:31.401083 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:31.401366 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:31.401369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:31.401759 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:31.401965 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:31.401974 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.402950 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.402953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:31.403874 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:31.403885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.404550 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.404553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:31.404679 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:31.405307 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:31.405324 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:31.408305 (+ 2981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":234,"threads_started":1}
W20260809 06:26:31.408742 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:31.400588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:34703 (local address 127.8.131.62:45267)
0809 06:26:31.400820 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:31.400827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:31.401050 (+ 223us) server_negotiation.cc:407] Connection header received
0809 06:26:31.401182 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:31.401186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:31.401246 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:31.401325 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:31.402097 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.402830 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.404007 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.404429 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.404519 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:31.408510 (+ 3991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:34703: BlockingRecv error: recv got EOF from 127.8.131.2:34703 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 06:26:31 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:31 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.3@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:31.889703 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:31.873200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.3:53277)
0809 06:26:31.881085 (+ 7885us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:31.881104 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:31.881243 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:31.881577 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:31.881581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:31.882083 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:31.882284 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:31.882294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.883330 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.883334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:31.884702 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:31.884761 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.885699 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.885704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:31.885869 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:31.886486 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:31.886505 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:31.889324 (+ 2819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":7765,"thread_start_us":7583,"threads_started":1}
W20260809 06:26:31.889809 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:31.873629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:53277 (local address 127.8.131.62:45267)
0809 06:26:31.873858 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:31.873866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:31.881201 (+ 7335us) server_negotiation.cc:407] Connection header received
0809 06:26:31.881366 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:31.881370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:31.881442 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:31.881526 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:31.882438 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.883167 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.884911 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.885541 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.885637 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:31.889583 (+ 3946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.3:53277: BlockingRecv error: recv got EOF from 127.8.131.3:53277 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 06:26:31 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:31 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.1@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:31.979363 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:31.971005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.1:42541)
0809 06:26:31.971573 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:31.971590 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:31.971766 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:31.972071 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:31.972074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:31.972503 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:31.972699 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:31.972708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.973706 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.973709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:31.974614 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:31.974625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.975263 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.975267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:31.975376 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:31.976061 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:31.976078 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:31.978982 (+ 2904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":314,"threads_started":1}
W20260809 06:26:31.979617 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:31.971090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:42541 (local address 127.8.131.62:45267)
0809 06:26:31.971310 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:31.971315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:31.971741 (+ 426us) server_negotiation.cc:407] Connection header received
0809 06:26:31.971884 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:31.971888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:31.971970 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:31.972057 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:31.972838 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.973581 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.974729 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:31.975142 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:31.975233 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:31.979177 (+ 3944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.1:42541: BlockingRecv error: recv got EOF from 127.8.131.1:42541 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 06:26:32 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 06:26:32 dist-test-slave-mjjr krb5kdc[21619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.131.2@KRBTEST.COM for oryx/127.8.131.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 06:26:32.418335 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:32.409985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:45267 (local address 127.8.131.2:39787)
0809 06:26:32.410529 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:32.410550 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:32.410656 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:32.410975 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:32.410978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:32.411397 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:32.411604 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:32.411613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.412691 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.412696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:32.413763 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:32.413776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.414577 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.414581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:32.414703 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:32.415283 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:32.415300 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:32.418083 (+ 2783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.131.62:45267: Server oryx/127.8.131.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":314,"threads_started":1}
W20260809 06:26:32.418574 22688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:32.410048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:39787 (local address 127.8.131.62:45267)
0809 06:26:32.410245 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:32.410252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:32.410626 (+ 374us) server_negotiation.cc:407] Connection header received
0809 06:26:32.410758 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:32.410762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:32.410843 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:32.410925 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:32.411729 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.412464 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.413900 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.414451 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.414550 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:32.418315 (+ 3765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.131.2:39787: BlockingRecv error: recv got EOF from 127.8.131.2:39787 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:26:32.709939 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 22481
I20260809 06:26:32.722332 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:32.711036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.8.131.1:43931)
0809 06:26:32.711567 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:32.711575 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:32.711604 (+ 29us) server_negotiation.cc:407] Connection header received
0809 06:26:32.711692 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:32.711700 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:32.711944 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:32.712113 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:32.713507 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.714689 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.716018 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.716585 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.716701 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:32.719171 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:32.719178 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:32.719192 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:32.719224 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:32.721845 (+ 2621us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.131.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 06:26:32.722101 (+ 256us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34194: Request ticket server kudu/127.8.131.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":239,"threads_started":1}
W20260809 06:26:32.722456 23136 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34194: Request ticket server kudu/127.8.131.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 06:26:32.722514 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 22481: Not authorized: Client connection negotiation failed: client connection to 127.8.131.1:43931: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.131.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 06:26:32.722769 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 22481
I20260809 06:26:32.745687 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 22691
I20260809 06:26:32.759948 23138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:32.746950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.8.131.2:33515)
0809 06:26:32.747574 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:32.747583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:32.747617 (+ 34us) server_negotiation.cc:407] Connection header received
0809 06:26:32.747766 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:32.747773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:32.748011 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:32.748218 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:32.749673 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.751430 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.752895 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.753507 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.753658 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:32.756407 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:32.756416 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:32.756430 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:32.756468 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:32.759125 (+ 2657us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.131.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 06:26:32.759518 (+ 393us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35644: Request ticket server kudu/127.8.131.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":291,"threads_started":1}
W20260809 06:26:32.760030 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 22691: Not authorized: Client connection negotiation failed: client connection to 127.8.131.2:33515: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.131.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 06:26:32.760076 23138 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35644: Request ticket server kudu/127.8.131.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 06:26:32.760174 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 22691
I20260809 06:26:32.786567 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 22873
I20260809 06:26:32.799364 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:32.787700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.8.131.3:40019)
0809 06:26:32.788325 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:32.788332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:32.788360 (+ 28us) server_negotiation.cc:407] Connection header received
0809 06:26:32.788421 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:32.788428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:32.788657 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:32.788807 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:32.790281 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.791385 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.792759 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:32.793358 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:32.793474 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:32.796151 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:32.796159 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:32.796173 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:32.796211 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:32.798830 (+ 2619us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.131.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 06:26:32.799118 (+ 288us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37166: Request ticket server kudu/127.8.131.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":336,"threads_started":1}
W20260809 06:26:32.799497 23139 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37166: Request ticket server kudu/127.8.131.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 06:26:32.799559 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 22873: Not authorized: Client connection negotiation failed: client connection to 127.8.131.3:40019: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.131.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 06:26:32.799800 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 22873
I20260809 06:26:32.823494 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 22366
I20260809 06:26:32.825076 22470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:32.945003 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 22366
2026-08-09T06:26:32Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21503 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T06:26:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:26:33Z Disabled control of system clock
I20260809 06:26:33.038962 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:44151
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:44151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 06:26:33.375981 23144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:33.376278 23144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:33.376358 23144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:33.383479 23144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:26:33.383586 23144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:33.383625 23144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:33.383750 23144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:26:33.383810 23144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:26:33.392961 23144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:44151
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:44151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:33.394215 23144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:33.395933 23144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:33.408404 23151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:33.408478 23155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:33.408452 23152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:33.408952 23144 server_base.cc:1034] running on GCE node
I20260809 06:26:33.409885 23144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:33.411283 23144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:33.412465 23144 hybrid_clock.cc:648] HybridClock initialized: now 1786256793412432 us; error 52 us; skew 500 ppm
I20260809 06:26:33.412832 23144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:26:33.415026 23144 webserver.cc:460] Webserver started at http://127.8.131.62:37361/ using document root <none> and password file <none>
I20260809 06:26:33.415791 23144 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:33.415915 23144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:33.416261 23144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:33.419168 23144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "1f430d270b284227b8555736741853d7"
format_stamp: "Formatted at 2026-08-09 06:26:33 on dist-test-slave-mjjr"
server_key: "de1ead5138ce671d782c52cbc03f3245"
server_key_iv: "ee8a2c9c316430a372a04bf760e0b308"
server_key_version: "encryptionkey@0"
I20260809 06:26:33.420053 23144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "1f430d270b284227b8555736741853d7"
format_stamp: "Formatted at 2026-08-09 06:26:33 on dist-test-slave-mjjr"
server_key: "de1ead5138ce671d782c52cbc03f3245"
server_key_iv: "ee8a2c9c316430a372a04bf760e0b308"
server_key_version: "encryptionkey@0"
I20260809 06:26:33.425161 23144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:26:33.428515 23162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:33.429804 23144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:26:33.429998 23144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "1f430d270b284227b8555736741853d7"
format_stamp: "Formatted at 2026-08-09 06:26:33 on dist-test-slave-mjjr"
server_key: "de1ead5138ce671d782c52cbc03f3245"
server_key_iv: "ee8a2c9c316430a372a04bf760e0b308"
server_key_version: "encryptionkey@0"
I20260809 06:26:33.430229 23144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:33.447279 23144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:33.448210 23144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:33.448510 23144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:33.467557 23144 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:44151
I20260809 06:26:33.467582 23237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:44151 every 8 connection(s)
I20260809 06:26:33.468952 23144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:26:33.472947 23238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:33.473946 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 23144
I20260809 06:26:33.474159 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:26:33.474838 8716 external_mini_cluster.cc:1428] Setting key f434877b12e44d37520678e1ea15186f
I20260809 06:26:33.489180 23238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7: Bootstrap starting.
I20260809 06:26:33.491492 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:33.476897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40886 (local address 127.8.131.62:44151)
0809 06:26:33.478122 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:33.478153 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:33.478210 (+ 57us) server_negotiation.cc:407] Connection header received
0809 06:26:33.480333 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:33.480362 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:33.481000 (+ 638us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:33.481487 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:33.482068 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:33.484253 (+ 2185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:33.485608 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:33.486836 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:33.487059 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:33.488426 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:33.488440 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:33.488459 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:33.489256 (+ 797us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:33.489374 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:33.489403 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:33.489536 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:33.489750 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 06:26:33.490075 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":357,"threads_started":1}
I20260809 06:26:33.493705 23238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:33.494977 23238 log.cc:824] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:33.498232 23238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7: No bootstrap required, opened a new log
I20260809 06:26:33.506543 23238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f430d270b284227b8555736741853d7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44151 } }
I20260809 06:26:33.506907 23238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:33.506989 23238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f430d270b284227b8555736741853d7, State: Initialized, Role: FOLLOWER
I20260809 06:26:33.507515 23238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f430d270b284227b8555736741853d7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44151 } }
I20260809 06:26:33.507812 23238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:26:33.507928 23238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:26:33.508064 23238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:33.510317 23238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f430d270b284227b8555736741853d7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44151 } }
I20260809 06:26:33.510852 23238 leader_election.cc:302] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f430d270b284227b8555736741853d7; no voters:
I20260809 06:26:33.511471 23238 leader_election.cc:288] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:26:33.511677 23243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:33.512332 23243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1f430d270b284227b8555736741853d7, State: Running, Role: LEADER
I20260809 06:26:33.513016 23243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f430d270b284227b8555736741853d7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44151 } }
I20260809 06:26:33.514346 23238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:26:33.516158 23244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f430d270b284227b8555736741853d7. Latest consensus state: current_term: 1 leader_uuid: "1f430d270b284227b8555736741853d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f430d270b284227b8555736741853d7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44151 } } }
I20260809 06:26:33.516637 23244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:33.516158 23245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f430d270b284227b8555736741853d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f430d270b284227b8555736741853d7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44151 } } }
I20260809 06:26:33.517211 23245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7 [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:33.519083 23250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:26:33.525476 23250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:26:33.534823 23250 catalog_manager.cc:1347] Generated new cluster ID: 08640c52665a45edac725b5a2b80ff2d
I20260809 06:26:33.534950 23250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:26:33.544772 23250 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:26:33.546072 23250 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:26:33.552515 23250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f430d270b284227b8555736741853d7: Generated new TSK 0
I20260809 06:26:33.553282 23250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:26:33.562597 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44151
--builtin_ntp_servers=127.8.131.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 06:26:33.936437 23268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:33.936736 23268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:33.936831 23268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:33.944199 23268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:33.944295 23268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:33.944396 23268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:26:33.954187 23268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:33.955544 23268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:33.957409 23268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:33.970422 23278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:33.970652 23276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:33.970652 23281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:33.971412 23268 server_base.cc:1034] running on GCE node
I20260809 06:26:33.972051 23268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:33.972995 23268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:33.974226 23268 hybrid_clock.cc:648] HybridClock initialized: now 1786256793974190 us; error 51 us; skew 500 ppm
I20260809 06:26:33.974543 23268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:26:33.976886 23268 webserver.cc:460] Webserver started at http://127.8.131.1:42097/ using document root <none> and password file <none>
I20260809 06:26:33.977578 23268 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:33.977712 23268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:33.978044 23268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:33.980882 23268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "124612453fd2422eafe47acc8b3548eb"
format_stamp: "Formatted at 2026-08-09 06:26:33 on dist-test-slave-mjjr"
server_key: "4b9b1450e2d53c87bd11c9f013996183"
server_key_iv: "b5a4a6a769f3397afaeab23ef9b3575d"
server_key_version: "encryptionkey@0"
I20260809 06:26:33.981688 23268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "124612453fd2422eafe47acc8b3548eb"
format_stamp: "Formatted at 2026-08-09 06:26:33 on dist-test-slave-mjjr"
server_key: "4b9b1450e2d53c87bd11c9f013996183"
server_key_iv: "b5a4a6a769f3397afaeab23ef9b3575d"
server_key_version: "encryptionkey@0"
I20260809 06:26:33.986701 23268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 06:26:33.990065 23288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:33.991423 23268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:26:33.991595 23268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "124612453fd2422eafe47acc8b3548eb"
format_stamp: "Formatted at 2026-08-09 06:26:33 on dist-test-slave-mjjr"
server_key: "4b9b1450e2d53c87bd11c9f013996183"
server_key_iv: "b5a4a6a769f3397afaeab23ef9b3575d"
server_key_version: "encryptionkey@0"
I20260809 06:26:33.991972 23268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:34.009070 23268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:34.009894 23268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:34.010178 23268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:34.011130 23268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:34.012616 23268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:34.012727 23268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:34.012858 23268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:34.012969 23268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:34.046371 23268 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:40441
I20260809 06:26:34.046448 23435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:40441 every 8 connection(s)
I20260809 06:26:34.047892 23268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:26:34.054459 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 23268
I20260809 06:26:34.054675 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:26:34.055323 8716 external_mini_cluster.cc:1428] Setting key 61b13e7ac8ff16ad973be3da39b34ba9
I20260809 06:26:34.058072 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44151
--builtin_ntp_servers=127.8.131.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 06:26:34.061142 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:34.050386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:36023 (local address 127.8.131.62:44151)
0809 06:26:34.050854 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:34.050859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:34.052168 (+ 1309us) server_negotiation.cc:407] Connection header received
0809 06:26:34.054006 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:34.054012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:34.054105 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:34.054225 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:34.055511 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.056441 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.058152 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.058631 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.058771 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:34.060229 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:34.060234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:34.060237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:34.060255 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:34.060298 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:34.060310 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:34.060427 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:34.060660 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 06:26:34.060760 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":243,"threads_started":1}
I20260809 06:26:34.062155 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:34.050995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:44151 (local address 127.8.131.1:36023)
0809 06:26:34.052001 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:34.052038 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:34.053698 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:34.054479 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:34.054487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:34.054640 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:34.055292 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:34.055308 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.056589 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.056595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:34.057988 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:34.058006 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.058871 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.058879 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:34.059127 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:34.059906 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:26:34.059909 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:34.059949 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:26:34.060026 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:34.060428 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:34.060443 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:34.060454 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 06:26:34.060644 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 06:26:34.061069 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":283,"threads_started":1}
I20260809 06:26:34.063822 23436 heartbeater.cc:347] Connected to a master server at 127.8.131.62:44151
I20260809 06:26:34.064158 23436 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:34.065142 23436 heartbeater.cc:511] Master 127.8.131.62:44151 requested a full tablet report, sending...
I20260809 06:26:34.067562 23189 ts_manager.cc:194] Registered new tserver with Master: 124612453fd2422eafe47acc8b3548eb (127.8.131.1:40441)
I20260809 06:26:34.069208 23189 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.1:36023
W20260809 06:26:34.401006 23446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:34.401319 23446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:34.401463 23446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:34.409797 23446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:34.409895 23446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:34.410055 23446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:26:34.419165 23446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:34.420557 23446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:34.422279 23446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:34.435233 23458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:34.435297 23455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:34.435881 23452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:34.436919 23446 server_base.cc:1034] running on GCE node
I20260809 06:26:34.437539 23446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:34.438400 23446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:34.439630 23446 hybrid_clock.cc:648] HybridClock initialized: now 1786256794439593 us; error 50 us; skew 500 ppm
I20260809 06:26:34.440004 23446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:26:34.442188 23446 webserver.cc:460] Webserver started at http://127.8.131.2:40157/ using document root <none> and password file <none>
I20260809 06:26:34.442873 23446 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:34.443004 23446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:34.443362 23446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:34.446166 23446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "dbb8b0233792458cbbfc80a10d44e6ef"
format_stamp: "Formatted at 2026-08-09 06:26:34 on dist-test-slave-mjjr"
server_key: "8632bde1f3dec94d94426318aa22e8eb"
server_key_iv: "9c2471516fb003c16fe00a8a50938a76"
server_key_version: "encryptionkey@0"
I20260809 06:26:34.446991 23446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "dbb8b0233792458cbbfc80a10d44e6ef"
format_stamp: "Formatted at 2026-08-09 06:26:34 on dist-test-slave-mjjr"
server_key: "8632bde1f3dec94d94426318aa22e8eb"
server_key_iv: "9c2471516fb003c16fe00a8a50938a76"
server_key_version: "encryptionkey@0"
I20260809 06:26:34.452068 23446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 06:26:34.455319 23467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:34.456666 23446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:26:34.456872 23446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "dbb8b0233792458cbbfc80a10d44e6ef"
format_stamp: "Formatted at 2026-08-09 06:26:34 on dist-test-slave-mjjr"
server_key: "8632bde1f3dec94d94426318aa22e8eb"
server_key_iv: "9c2471516fb003c16fe00a8a50938a76"
server_key_version: "encryptionkey@0"
I20260809 06:26:34.457086 23446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:34.474272 23446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:34.475086 23446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:34.475368 23446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:34.476395 23446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:34.477893 23446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:34.478019 23446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:34.478157 23446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:34.478245 23446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:34.512300 23446 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:32805
I20260809 06:26:34.512351 23619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:32805 every 8 connection(s)
I20260809 06:26:34.513582 23446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:26:34.516001 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 23446
I20260809 06:26:34.516192 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:26:34.516733 8716 external_mini_cluster.cc:1428] Setting key ac1897cbd9f4e367be6849328008c2c1
I20260809 06:26:34.519467 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44151
--builtin_ntp_servers=127.8.131.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 06:26:34.526980 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:34.516338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:55455 (local address 127.8.131.62:44151)
0809 06:26:34.516600 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:34.516606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:34.518214 (+ 1608us) server_negotiation.cc:407] Connection header received
0809 06:26:34.520127 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:34.520133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:34.520215 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:34.520335 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:34.521424 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.522192 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.523774 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.524180 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.524317 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:34.525676 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:34.525681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:34.525684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:34.525703 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:34.525790 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:34.525812 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:34.525904 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:34.526247 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 06:26:34.526420 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 06:26:34.527927 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:34.516987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:44151 (local address 127.8.131.2:55455)
0809 06:26:34.518064 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:34.518110 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:34.519872 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:34.520567 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:34.520575 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:34.520682 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:34.521262 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:34.521278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.522353 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.522358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:34.523584 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:34.523595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.524418 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.524426 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:34.524634 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:34.525327 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:26:34.525330 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:34.525376 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:26:34.525450 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:34.525960 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:34.525973 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:34.526003 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 06:26:34.526194 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 06:26:34.526593 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":374,"threads_started":1}
I20260809 06:26:34.529321 23620 heartbeater.cc:347] Connected to a master server at 127.8.131.62:44151
I20260809 06:26:34.529660 23620 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:34.530599 23620 heartbeater.cc:511] Master 127.8.131.62:44151 requested a full tablet report, sending...
I20260809 06:26:34.532172 23189 ts_manager.cc:194] Registered new tserver with Master: dbb8b0233792458cbbfc80a10d44e6ef (127.8.131.2:32805)
I20260809 06:26:34.532986 23189 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.2:55455
W20260809 06:26:34.854149 23626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:34.854419 23626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:34.854717 23626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:34.862192 23626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:34.862282 23626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:34.862478 23626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:26:34.871534 23626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:34.872851 23626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:34.874630 23626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:34.888278 23631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:34.888240 23632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:34.888334 23635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:34.889194 23626 server_base.cc:1034] running on GCE node
I20260809 06:26:34.889808 23626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:34.890671 23626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:34.891865 23626 hybrid_clock.cc:648] HybridClock initialized: now 1786256794891794 us; error 58 us; skew 500 ppm
I20260809 06:26:34.892165 23626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:26:34.894412 23626 webserver.cc:460] Webserver started at http://127.8.131.3:40477/ using document root <none> and password file <none>
I20260809 06:26:34.895161 23626 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:34.895269 23626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:34.895625 23626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:34.898494 23626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "65628c8bf9fa4e06814adb3e3d16349f"
format_stamp: "Formatted at 2026-08-09 06:26:34 on dist-test-slave-mjjr"
server_key: "1b43520e35af267ca441f0d0f4f59bf8"
server_key_iv: "0817016dd8ff7872c7aad771d9846608"
server_key_version: "encryptionkey@0"
I20260809 06:26:34.899300 23626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "65628c8bf9fa4e06814adb3e3d16349f"
format_stamp: "Formatted at 2026-08-09 06:26:34 on dist-test-slave-mjjr"
server_key: "1b43520e35af267ca441f0d0f4f59bf8"
server_key_iv: "0817016dd8ff7872c7aad771d9846608"
server_key_version: "encryptionkey@0"
I20260809 06:26:34.904213 23626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 06:26:34.907554 23644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:34.908901 23626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:26:34.909085 23626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "65628c8bf9fa4e06814adb3e3d16349f"
format_stamp: "Formatted at 2026-08-09 06:26:34 on dist-test-slave-mjjr"
server_key: "1b43520e35af267ca441f0d0f4f59bf8"
server_key_iv: "0817016dd8ff7872c7aad771d9846608"
server_key_version: "encryptionkey@0"
I20260809 06:26:34.909305 23626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:34.921826 23626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:34.922629 23626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:34.922923 23626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:34.923949 23626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:34.925345 23626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:34.925468 23626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:34.925602 23626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:34.925690 23626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:34.961443 23626 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:33533
I20260809 06:26:34.961483 23800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:33533 every 8 connection(s)
I20260809 06:26:34.962826 23626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:26:34.969229 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 23626
I20260809 06:26:34.969434 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:26:34.969909 8716 external_mini_cluster.cc:1428] Setting key 316978241f850c568e6bdafadedfb1d2
I20260809 06:26:34.975471 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:34.965603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:49045 (local address 127.8.131.62:44151)
0809 06:26:34.965796 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:34.965800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:34.967244 (+ 1444us) server_negotiation.cc:407] Connection header received
0809 06:26:34.969030 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:34.969036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:34.969129 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:34.969246 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:34.970435 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.971193 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.972789 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.973185 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.973328 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:34.974591 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:34.974597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:34.974608 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:34.974624 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:34.974655 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:34.974666 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:34.974762 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:34.975000 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 06:26:34.975114 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 06:26:34.976447 23804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:34.966089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:44151 (local address 127.8.131.3:49045)
0809 06:26:34.967075 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:34.967111 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:34.968744 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:34.969480 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:34.969487 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:34.969621 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:34.970242 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:34.970260 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.971356 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.971364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:34.972642 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:34.972654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.973419 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.973426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:34.973657 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:34.974297 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:26:34.974299 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:34.974341 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:26:34.974411 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:34.974761 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:34.974774 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:34.974784 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 06:26:34.974994 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 06:26:34.975375 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":350,"threads_started":1}
I20260809 06:26:34.977845 23802 heartbeater.cc:347] Connected to a master server at 127.8.131.62:44151
I20260809 06:26:34.978176 23802 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:34.978914 23802 heartbeater.cc:511] Master 127.8.131.62:44151 requested a full tablet report, sending...
I20260809 06:26:34.980322 23189 ts_manager.cc:194] Registered new tserver with Master: 65628c8bf9fa4e06814adb3e3d16349f (127.8.131.3:33533)
I20260809 06:26:34.981058 23189 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.3:49045
I20260809 06:26:34.985033 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 06:26:34.991220 23440 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:34.989494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40898 (local address 127.8.131.62:44151)
0809 06:26:34.989746 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:34.989754 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:34.990104 (+ 350us) server_negotiation.cc:407] Connection header received
0809 06:26:34.990300 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:34.990307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:34.990390 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:34.990494 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:34.990979 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40898: BlockingRecv error: recv got EOF from 127.0.0.1:40898 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 06:26:34.992137 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 23268
I20260809 06:26:35.000247 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:34.993115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.8.131.1:40441)
0809 06:26:34.993705 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:34.993711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:34.993736 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:26:34.993798 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:34.993804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:34.993989 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:34.994116 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:34.994463 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.995625 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.997019 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:34.997606 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:34.997733 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:34.998704 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:34.998712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:34.998726 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:34.999368 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:34.999416 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:34.999436 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:34.999527 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:34.999626 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 06:26:34.999865 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":312,"threads_started":1}
I20260809 06:26:35.001482 23431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:35.082333 23436 heartbeater.cc:503] Master 127.8.131.62:44151 was elected leader, sending a full tablet report...
I20260809 06:26:35.083709 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 23268
W20260809 06:26:35.103503 23167 connection.cc:535] server connection from 127.8.131.1:36023 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 06:26:35.103775 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 23446
I20260809 06:26:35.111554 23817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:35.104752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38740 (local address 127.8.131.2:32805)
0809 06:26:35.105286 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:35.105293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:35.105319 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:26:35.105385 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:35.105391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:35.105612 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:35.105741 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:35.106114 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:35.107135 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:35.108565 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:35.109109 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:35.109201 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:35.110189 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:35.110196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:35.110208 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:35.110762 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:35.110807 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:35.110824 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:35.110920 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:35.111051 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 06:26:35.111200 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":266,"threads_started":1}
I20260809 06:26:35.112775 23613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:35.193667 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 23446
I20260809 06:26:35.216303 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 23626
I20260809 06:26:35.224597 23819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:35.217358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35206 (local address 127.8.131.3:33533)
0809 06:26:35.218042 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:35.218049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:35.218080 (+ 31us) server_negotiation.cc:407] Connection header received
0809 06:26:35.218138 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:35.218144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:35.218360 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:35.218493 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:35.218924 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:35.220163 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:35.221410 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:35.221992 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:35.222087 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:35.223063 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:35.223071 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:35.223083 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:35.223723 (+ 640us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:35.223784 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:35.223817 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:35.223926 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:35.224057 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 06:26:35.224210 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":359,"threads_started":1}
I20260809 06:26:35.225908 23796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:35.308208 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 23626
I20260809 06:26:35.340785 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 23144
I20260809 06:26:35.342159 23233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:35.450475 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 23144
2026-08-09T06:26:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2501 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T06:26:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:26:35Z Disabled control of system clock
I20260809 06:26:35.539377 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:33637
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:33637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 06:26:35.875782 23829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:35.876080 23829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:35.876173 23829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:35.883445 23829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:26:35.883553 23829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:35.883594 23829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:35.883699 23829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:26:35.883760 23829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:26:35.893079 23829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:33637
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:33637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:35.894361 23829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:35.896193 23829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:35.907959 23837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:35.907997 23835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:35.907918 23839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:35.909092 23829 server_base.cc:1034] running on GCE node
I20260809 06:26:35.909873 23829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:35.911274 23829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:35.912483 23829 hybrid_clock.cc:648] HybridClock initialized: now 1786256795912423 us; error 49 us; skew 500 ppm
I20260809 06:26:35.912793 23829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:26:35.914884 23829 webserver.cc:460] Webserver started at http://127.8.131.62:39577/ using document root <none> and password file <none>
I20260809 06:26:35.915508 23829 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:35.915593 23829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:35.915940 23829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:35.918720 23829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "4e48fe59e0104a16b8ea7d5bf730a55c"
format_stamp: "Formatted at 2026-08-09 06:26:35 on dist-test-slave-mjjr"
server_key: "a8be648c30f922b358258c23e48087b3"
server_key_iv: "5263dadfcb3982c3037af57280bc604d"
server_key_version: "encryptionkey@0"
I20260809 06:26:35.919463 23829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "4e48fe59e0104a16b8ea7d5bf730a55c"
format_stamp: "Formatted at 2026-08-09 06:26:35 on dist-test-slave-mjjr"
server_key: "a8be648c30f922b358258c23e48087b3"
server_key_iv: "5263dadfcb3982c3037af57280bc604d"
server_key_version: "encryptionkey@0"
I20260809 06:26:35.924614 23829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:26:35.927942 23848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:35.929221 23829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:26:35.929402 23829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "4e48fe59e0104a16b8ea7d5bf730a55c"
format_stamp: "Formatted at 2026-08-09 06:26:35 on dist-test-slave-mjjr"
server_key: "a8be648c30f922b358258c23e48087b3"
server_key_iv: "5263dadfcb3982c3037af57280bc604d"
server_key_version: "encryptionkey@0"
I20260809 06:26:35.929618 23829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:35.935258 23829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:35.936146 23829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:35.936456 23829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:35.954110 23829 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:33637
I20260809 06:26:35.954142 23931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:33637 every 8 connection(s)
I20260809 06:26:35.955523 23829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:26:35.959367 23933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:35.962119 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 23829
I20260809 06:26:35.962345 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:26:35.963174 8716 external_mini_cluster.cc:1428] Setting key 82944ea61ad30899720fa609ceaaad99
I20260809 06:26:35.971689 23936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:35.965389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33840 (local address 127.8.131.62:33637)
0809 06:26:35.966384 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:35.966394 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:35.966436 (+ 42us) server_negotiation.cc:407] Connection header received
0809 06:26:35.968161 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:35.968182 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:35.968606 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:35.968925 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:35.970066 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:35.970072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:35.970083 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:35.970099 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:35.970155 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:35.970170 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:35.970346 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:35.970488 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 06:26:35.970774 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":400,"threads_started":1}
I20260809 06:26:35.972360 23933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c: Bootstrap starting.
I20260809 06:26:35.975570 23933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:35.976583 23933 log.cc:824] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:35.979449 23933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c: No bootstrap required, opened a new log
I20260809 06:26:35.986752 23933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e48fe59e0104a16b8ea7d5bf730a55c" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33637 } }
I20260809 06:26:35.987090 23933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:35.987164 23933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e48fe59e0104a16b8ea7d5bf730a55c, State: Initialized, Role: FOLLOWER
I20260809 06:26:35.987713 23933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e48fe59e0104a16b8ea7d5bf730a55c" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33637 } }
I20260809 06:26:35.987926 23933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:26:35.988025 23933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:26:35.988142 23933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:35.990187 23933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e48fe59e0104a16b8ea7d5bf730a55c" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33637 } }
I20260809 06:26:35.990633 23933 leader_election.cc:302] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e48fe59e0104a16b8ea7d5bf730a55c; no voters:
I20260809 06:26:35.991133 23933 leader_election.cc:288] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:26:35.991271 23938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:35.992120 23938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [term 1 LEADER]: Becoming Leader. State: Replica: 4e48fe59e0104a16b8ea7d5bf730a55c, State: Running, Role: LEADER
I20260809 06:26:35.992673 23938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e48fe59e0104a16b8ea7d5bf730a55c" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33637 } }
I20260809 06:26:35.993323 23933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:26:35.995810 23940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e48fe59e0104a16b8ea7d5bf730a55c. Latest consensus state: current_term: 1 leader_uuid: "4e48fe59e0104a16b8ea7d5bf730a55c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e48fe59e0104a16b8ea7d5bf730a55c" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33637 } } }
I20260809 06:26:35.996232 23940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:35.997025 23939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e48fe59e0104a16b8ea7d5bf730a55c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e48fe59e0104a16b8ea7d5bf730a55c" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33637 } } }
I20260809 06:26:35.997501 23939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:35.999899 23947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:26:36.005093 23947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:26:36.013635 23947 catalog_manager.cc:1347] Generated new cluster ID: 9e6cc2373b47463fae3946d15ca093e0
I20260809 06:26:36.013747 23947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:26:36.033166 23947 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:26:36.033650 23947 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:26:36.046979 23947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e48fe59e0104a16b8ea7d5bf730a55c: Generated new TSK 0
I20260809 06:26:36.047853 23947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:26:36.062464 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33637
--builtin_ntp_servers=127.8.131.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 06:26:36.414939 23960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:36.415248 23960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:36.415342 23960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:36.422541 23960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:36.422637 23960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:36.422744 23960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:26:36.431810 23960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:36.433009 23960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:36.434779 23960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:36.448213 23968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:36.448171 23970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:36.448246 23967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:36.448567 23960 server_base.cc:1034] running on GCE node
I20260809 06:26:36.449414 23960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:36.450320 23960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:36.451604 23960 hybrid_clock.cc:648] HybridClock initialized: now 1786256796451528 us; error 51 us; skew 500 ppm
I20260809 06:26:36.452032 23960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:26:36.454211 23960 webserver.cc:460] Webserver started at http://127.8.131.1:37483/ using document root <none> and password file <none>
I20260809 06:26:36.454928 23960 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:36.455055 23960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:36.455408 23960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:36.460882 23960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "a7a7197a3dee480c9139a040a3036138"
format_stamp: "Formatted at 2026-08-09 06:26:36 on dist-test-slave-mjjr"
server_key: "4ba9056a62f17dbc2f2aeef8c020ebf7"
server_key_iv: "fd06a6d4cb3fb3bc2ccabbc75a309f7d"
server_key_version: "encryptionkey@0"
I20260809 06:26:36.461727 23960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "a7a7197a3dee480c9139a040a3036138"
format_stamp: "Formatted at 2026-08-09 06:26:36 on dist-test-slave-mjjr"
server_key: "4ba9056a62f17dbc2f2aeef8c020ebf7"
server_key_iv: "fd06a6d4cb3fb3bc2ccabbc75a309f7d"
server_key_version: "encryptionkey@0"
I20260809 06:26:36.466866 23960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:26:36.470228 23978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:36.471719 23960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:26:36.471894 23960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "a7a7197a3dee480c9139a040a3036138"
format_stamp: "Formatted at 2026-08-09 06:26:36 on dist-test-slave-mjjr"
server_key: "4ba9056a62f17dbc2f2aeef8c020ebf7"
server_key_iv: "fd06a6d4cb3fb3bc2ccabbc75a309f7d"
server_key_version: "encryptionkey@0"
I20260809 06:26:36.472112 23960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:36.477662 23960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:36.478363 23960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:36.478608 23960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:36.479434 23960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:36.480809 23960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:36.480898 23960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:36.480978 23960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:36.481033 23960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:36.514220 23960 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:35537
I20260809 06:26:36.514294 24135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:35537 every 8 connection(s)
I20260809 06:26:36.515515 23960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:26:36.523517 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 23960
I20260809 06:26:36.523633 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:36.518050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:37587 (local address 127.8.131.62:33637)
0809 06:26:36.518501 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:36.518505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:36.519631 (+ 1126us) server_negotiation.cc:407] Connection header received
0809 06:26:36.521328 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:36.521334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:36.521415 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:36.521534 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:36.522846 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:36.522858 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:36.522861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:36.522875 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:36.522917 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:36.522929 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:36.523064 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:36.523248 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 06:26:36.523318 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":225,"threads_started":1}
I20260809 06:26:36.523780 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:26:36.524458 24144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:36.518606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:33637 (local address 127.8.131.1:37587)
0809 06:26:36.519471 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:36.519510 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:36.521065 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:36.521772 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:36.521780 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:36.521862 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:36.522566 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:26:36.522568 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:36.522602 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:26:36.522681 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:36.523025 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:36.523037 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:36.523047 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 06:26:36.523327 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 06:26:36.523556 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":275,"threads_started":1}
I20260809 06:26:36.524469 8716 external_mini_cluster.cc:1428] Setting key 61832f4048db57960500c4d2ea0ac1dd
I20260809 06:26:36.526081 24139 heartbeater.cc:347] Connected to a master server at 127.8.131.62:33637
I20260809 06:26:36.526365 24139 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:36.527050 24139 heartbeater.cc:511] Master 127.8.131.62:33637 requested a full tablet report, sending...
I20260809 06:26:36.527899 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33637
--builtin_ntp_servers=127.8.131.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 06:26:36.529174 23876 ts_manager.cc:194] Registered new tserver with Master: a7a7197a3dee480c9139a040a3036138 (127.8.131.1:35537)
W20260809 06:26:36.865839 24145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:36.866133 24145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:36.866303 24145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:36.873621 24145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:36.873756 24145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:36.873946 24145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:26:36.883114 24145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:36.884568 24145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:36.886368 24145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:36.900046 24150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:36.899992 24154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:36.899955 24151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:36.900533 24145 server_base.cc:1034] running on GCE node
I20260809 06:26:36.901365 24145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:36.902626 24145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:36.903928 24145 hybrid_clock.cc:648] HybridClock initialized: now 1786256796903854 us; error 54 us; skew 500 ppm
I20260809 06:26:36.904302 24145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:26:36.906780 24145 webserver.cc:460] Webserver started at http://127.8.131.2:39879/ using document root <none> and password file <none>
I20260809 06:26:36.907524 24145 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:36.907684 24145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:36.908058 24145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:36.911003 24145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "4dd96b1a2d934daaa635632fdb127b72"
format_stamp: "Formatted at 2026-08-09 06:26:36 on dist-test-slave-mjjr"
server_key: "71381742d7d9514f6a14f8b83084d4c2"
server_key_iv: "cd5c37c9eff5fb90b57d9ad691d7bd41"
server_key_version: "encryptionkey@0"
I20260809 06:26:36.911906 24145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "4dd96b1a2d934daaa635632fdb127b72"
format_stamp: "Formatted at 2026-08-09 06:26:36 on dist-test-slave-mjjr"
server_key: "71381742d7d9514f6a14f8b83084d4c2"
server_key_iv: "cd5c37c9eff5fb90b57d9ad691d7bd41"
server_key_version: "encryptionkey@0"
I20260809 06:26:36.917254 24145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:26:36.920830 24162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:36.922201 24145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:26:36.922395 24145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "4dd96b1a2d934daaa635632fdb127b72"
format_stamp: "Formatted at 2026-08-09 06:26:36 on dist-test-slave-mjjr"
server_key: "71381742d7d9514f6a14f8b83084d4c2"
server_key_iv: "cd5c37c9eff5fb90b57d9ad691d7bd41"
server_key_version: "encryptionkey@0"
I20260809 06:26:36.922632 24145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:36.928589 24145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:36.929385 24145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:36.929658 24145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:36.930599 24145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:36.932080 24145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:36.932205 24145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:36.932353 24145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:36.932444 24145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:36.972846 24145 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:41567
I20260809 06:26:36.972896 24316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:41567 every 8 connection(s)
I20260809 06:26:36.974283 24145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:26:36.975555 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 24145
I20260809 06:26:36.975904 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:26:36.976480 8716 external_mini_cluster.cc:1428] Setting key 5b123d68fdf37b65403ed2921aaefee8
I20260809 06:26:36.979806 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33637
--builtin_ntp_servers=127.8.131.20:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 06:26:36.984434 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:36.977435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:45831 (local address 127.8.131.62:33637)
0809 06:26:36.977672 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:36.977680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:36.979311 (+ 1631us) server_negotiation.cc:407] Connection header received
0809 06:26:36.982040 (+ 2729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:36.982044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:36.982096 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:36.982194 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:36.983764 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:36.983768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:36.983772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:36.983788 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:36.983822 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:36.983833 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:36.983933 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:36.984159 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 06:26:36.984211 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 06:26:36.985702 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:36.978030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:33637 (local address 127.8.131.2:45831)
0809 06:26:36.979135 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:36.979180 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:36.981729 (+ 2549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:36.982462 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:36.982476 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:36.982600 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:36.983328 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:26:36.983333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:36.983384 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:26:36.983496 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:36.983939 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:36.983952 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:36.983962 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 06:26:36.984238 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 06:26:36.984498 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":346,"threads_started":1}
I20260809 06:26:36.987181 24318 heartbeater.cc:347] Connected to a master server at 127.8.131.62:33637
I20260809 06:26:36.987505 24318 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:36.988270 24318 heartbeater.cc:511] Master 127.8.131.62:33637 requested a full tablet report, sending...
I20260809 06:26:36.989936 23876 ts_manager.cc:194] Registered new tserver with Master: 4dd96b1a2d934daaa635632fdb127b72 (127.8.131.2:41567)
W20260809 06:26:37.310874 24324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:37.311184 24324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:37.311352 24324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:37.318463 24324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:37.318607 24324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:37.318785 24324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:26:37.327870 24324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:37.329144 24324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:37.330883 24324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:37.344156 24333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:37.344223 24330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:37.344251 24329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:37.345193 24324 server_base.cc:1034] running on GCE node
I20260809 06:26:37.345798 24324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:37.346683 24324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:37.347905 24324 hybrid_clock.cc:648] HybridClock initialized: now 1786256797347847 us; error 50 us; skew 500 ppm
I20260809 06:26:37.348212 24324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:26:37.350387 24324 webserver.cc:460] Webserver started at http://127.8.131.3:43793/ using document root <none> and password file <none>
I20260809 06:26:37.351059 24324 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:37.351177 24324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:37.351542 24324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:37.354519 24324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "109e8fd8b64845a19f987c3baf1f4799"
format_stamp: "Formatted at 2026-08-09 06:26:37 on dist-test-slave-mjjr"
server_key: "349648c79306939ef9b3a94d4cd017ac"
server_key_iv: "ec96c2f6867cf2301e0e5ccc05c82ff4"
server_key_version: "encryptionkey@0"
I20260809 06:26:37.355319 24324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "109e8fd8b64845a19f987c3baf1f4799"
format_stamp: "Formatted at 2026-08-09 06:26:37 on dist-test-slave-mjjr"
server_key: "349648c79306939ef9b3a94d4cd017ac"
server_key_iv: "ec96c2f6867cf2301e0e5ccc05c82ff4"
server_key_version: "encryptionkey@0"
I20260809 06:26:37.360620 24324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:26:37.364089 24343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:37.365329 24324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:26:37.365511 24324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "109e8fd8b64845a19f987c3baf1f4799"
format_stamp: "Formatted at 2026-08-09 06:26:37 on dist-test-slave-mjjr"
server_key: "349648c79306939ef9b3a94d4cd017ac"
server_key_iv: "ec96c2f6867cf2301e0e5ccc05c82ff4"
server_key_version: "encryptionkey@0"
I20260809 06:26:37.365722 24324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:37.371425 24324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:37.372243 24324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:37.372524 24324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:37.373435 24324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:37.374804 24324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:37.374917 24324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:37.375080 24324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:37.375160 24324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:37.408264 24324 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:40733
I20260809 06:26:37.408344 24495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:40733 every 8 connection(s)
I20260809 06:26:37.409679 24324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:26:37.417738 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:37.412319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:39665 (local address 127.8.131.62:33637)
0809 06:26:37.412500 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:37.412504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:37.413778 (+ 1274us) server_negotiation.cc:407] Connection header received
0809 06:26:37.415454 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:37.415460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:37.415543 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:37.415668 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:37.416966 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:37.416972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:37.416978 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:37.417004 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:37.417036 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:37.417049 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:37.417180 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:37.417372 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 06:26:37.417449 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:26:37.418603 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:37.412718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:33637 (local address 127.8.131.3:39665)
0809 06:26:37.413614 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:37.413653 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:37.415178 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:37.415896 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:37.415903 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:37.415988 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:37.416667 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:26:37.416669 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:37.416706 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:26:37.416786 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:37.417146 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:37.417157 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:37.417166 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 06:26:37.417463 (+ 297us) client_negotiation.cc:240] Negotiation successful
0809 06:26:37.417708 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":290,"threads_started":1}
I20260809 06:26:37.418876 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 24324
I20260809 06:26:37.419113 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:26:37.419764 8716 external_mini_cluster.cc:1428] Setting key 1ebc62edb92cb9b4d399836766fa3d86
I20260809 06:26:37.420143 24496 heartbeater.cc:347] Connected to a master server at 127.8.131.62:33637
I20260809 06:26:37.420455 24496 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:37.421116 24496 heartbeater.cc:511] Master 127.8.131.62:33637 requested a full tablet report, sending...
I20260809 06:26:37.422610 23877 ts_manager.cc:194] Registered new tserver with Master: 109e8fd8b64845a19f987c3baf1f4799 (127.8.131.3:40733)
I20260809 06:26:37.434968 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 06:26:37.440863 24143 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:26:37.439417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33846 (local address 127.8.131.62:33637)
0809 06:26:37.439581 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:37.439586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:37.440133 (+ 547us) server_negotiation.cc:407] Connection header received
0809 06:26:37.440262 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:37.440266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:37.440319 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:37.440409 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:37.440676 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33846: BlockingRecv error: recv got EOF from 127.0.0.1:33846 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 06:26:37.441437 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 23960
I20260809 06:26:37.445204 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:37.442561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.8.131.1:35537)
0809 06:26:37.443129 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:37.443136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:37.443161 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:26:37.443216 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:37.443222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:37.443424 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:37.443549 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:37.444625 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:37.444632 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:37.444643 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:37.444661 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:37.444703 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:37.444717 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:37.444820 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:37.444908 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 06:26:37.444983 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":289,"threads_started":1}
I20260809 06:26:37.446429 24130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:37.528697 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 23960
I20260809 06:26:37.549393 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 24145
I20260809 06:26:37.553143 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:37.550400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36786 (local address 127.8.131.2:41567)
0809 06:26:37.550974 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:37.550982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:37.551013 (+ 31us) server_negotiation.cc:407] Connection header received
0809 06:26:37.551090 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:37.551097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:37.551336 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:37.551477 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:37.552400 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:37.552407 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:37.552418 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:37.552432 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:37.552480 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:37.552501 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:37.552630 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:37.552700 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 06:26:37.552829 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":269,"threads_started":1}
I20260809 06:26:37.554423 24312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:37.636541 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 24145
I20260809 06:26:37.658227 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 24324
I20260809 06:26:37.662420 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:37.659244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35120 (local address 127.8.131.3:40733)
0809 06:26:37.659979 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:37.659986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:37.660018 (+ 32us) server_negotiation.cc:407] Connection header received
0809 06:26:37.660081 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:37.660087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:37.660301 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:37.660471 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:37.661648 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:37.661655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:37.661666 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:26:37.661683 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:37.661721 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:26:37.661734 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:37.661931 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:37.662007 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 06:26:37.662110 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":341,"threads_started":1}
I20260809 06:26:37.663620 24489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:37.747033 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 24324
I20260809 06:26:37.769198 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 23829
I20260809 06:26:37.770538 23926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:37.879177 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 23829
2026-08-09T06:26:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2428 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:26:37 dist-test-slave-mjjr krb5kdc[24522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:26:37 dist-test-slave-mjjr krb5kdc[24522](info): set up 2 sockets
Aug 09 06:26:37 dist-test-slave-mjjr krb5kdc[24522](info): commencing operation
krb5kdc: starting...
W20260809 06:26:39.189839 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.003s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:26:39 dist-test-slave-mjjr krb5kdc[24522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:26:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:26:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:26:39.457226 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:44407
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:44407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:39.794305 24542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:39.794605 24542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:39.794785 24542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:39.802013 24542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:26:39.802124 24542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:39.802245 24542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:39.802350 24542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:26:39.802438 24542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:26:39.812319 24542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:44407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:44407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:39.813619 24542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:39.814906 24542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:39.826990 24548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:39.827030 24553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:39.827268 24547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:39.828699 24542 server_base.cc:1034] running on GCE node
I20260809 06:26:39.829814 24542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:39.831343 24542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:39.832551 24542 hybrid_clock.cc:648] HybridClock initialized: now 1786256799832474 us; error 63 us; skew 500 ppm
I20260809 06:26:39.832885 24542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:39 dist-test-slave-mjjr krb5kdc[24522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256799, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:39.839011 24542 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:26:39.840512 24542 webserver.cc:460] Webserver started at http://127.8.131.62:33975/ using document root <none> and password file <none>
I20260809 06:26:39.841152 24542 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:39.841253 24542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:39.841503 24542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:39.844641 24542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "63b244fba13e4fff92d2bcb69a16c87e"
format_stamp: "Formatted at 2026-08-09 06:26:39 on dist-test-slave-mjjr"
server_key: "d267f4f735bc68af86855ec1a8e12e54"
server_key_iv: "228bcc81d5fa4f32d77ed3ea4cfbc45b"
server_key_version: "encryptionkey@0"
I20260809 06:26:39.845356 24542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "63b244fba13e4fff92d2bcb69a16c87e"
format_stamp: "Formatted at 2026-08-09 06:26:39 on dist-test-slave-mjjr"
server_key: "d267f4f735bc68af86855ec1a8e12e54"
server_key_iv: "228bcc81d5fa4f32d77ed3ea4cfbc45b"
server_key_version: "encryptionkey@0"
I20260809 06:26:39.850827 24542 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:26:39.854350 24565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:39.855809 24542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 06:26:39.856011 24542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "63b244fba13e4fff92d2bcb69a16c87e"
format_stamp: "Formatted at 2026-08-09 06:26:39 on dist-test-slave-mjjr"
server_key: "d267f4f735bc68af86855ec1a8e12e54"
server_key_iv: "228bcc81d5fa4f32d77ed3ea4cfbc45b"
server_key_version: "encryptionkey@0"
I20260809 06:26:39.856216 24542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:39.873298 24542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:39.877789 24542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:39.878087 24542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:39.896808 24542 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:44407
I20260809 06:26:39.896891 24642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:44407 every 8 connection(s)
I20260809 06:26:39.898208 24542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:26:39.901854 24643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:39.903525 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 24542
I20260809 06:26:39.903813 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:26:39.904484 8716 external_mini_cluster.cc:1428] Setting key f84ddedd1f964285acaf74eb82cb047e
I20260809 06:26:39.914100 24643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e: Bootstrap starting.
I20260809 06:26:39.917893 24643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:39.919171 24643 log.cc:824] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e: Log is configured to *not* fsync() on all Append() calls
Aug 09 06:26:39 dist-test-slave-mjjr krb5kdc[24522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:26:39.922091 24643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e: No bootstrap required, opened a new log
I20260809 06:26:39.928332 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:39.906494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35680 (local address 127.8.131.62:44407)
0809 06:26:39.907310 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:39.907321 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:39.907363 (+ 42us) server_negotiation.cc:407] Connection header received
0809 06:26:39.909056 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:39.909076 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:39.909439 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:39.909736 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:39.911863 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:39.913148 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:39.914945 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:39.915594 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:39.915816 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:39.920843 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:39.920851 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:39.920864 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:39.920896 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:39.924058 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:39.924742 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:39.924750 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:39.924755 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:39.924926 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:39.925328 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:39.925335 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:39.925343 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:39.925838 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:39.926201 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:39.926598 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 06:26:39.927021 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":345,"threads_started":1}
I20260809 06:26:39.930383 24643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b244fba13e4fff92d2bcb69a16c87e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44407 } }
I20260809 06:26:39.930722 24643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:39.930863 24643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b244fba13e4fff92d2bcb69a16c87e, State: Initialized, Role: FOLLOWER
I20260809 06:26:39.931475 24643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b244fba13e4fff92d2bcb69a16c87e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44407 } }
I20260809 06:26:39.931741 24643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:26:39.931891 24643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:26:39.932103 24643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:39.934603 24643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b244fba13e4fff92d2bcb69a16c87e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44407 } }
I20260809 06:26:39.935086 24643 leader_election.cc:302] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63b244fba13e4fff92d2bcb69a16c87e; no voters:
I20260809 06:26:39.935817 24643 leader_election.cc:288] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:26:39.935891 24650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:39.937083 24650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [term 1 LEADER]: Becoming Leader. State: Replica: 63b244fba13e4fff92d2bcb69a16c87e, State: Running, Role: LEADER
I20260809 06:26:39.937602 24650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b244fba13e4fff92d2bcb69a16c87e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44407 } }
I20260809 06:26:39.938222 24643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:26:39.940912 24651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63b244fba13e4fff92d2bcb69a16c87e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b244fba13e4fff92d2bcb69a16c87e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44407 } } }
I20260809 06:26:39.941244 24651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:39.941974 24652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63b244fba13e4fff92d2bcb69a16c87e. Latest consensus state: current_term: 1 leader_uuid: "63b244fba13e4fff92d2bcb69a16c87e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b244fba13e4fff92d2bcb69a16c87e" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44407 } } }
I20260809 06:26:39.942351 24652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e [sys.catalog]: This master's current role is: LEADER
I20260809 06:26:39.942785 24658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:26:39.948410 24658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:26:39.958627 24658 catalog_manager.cc:1347] Generated new cluster ID: 5596ca8757704e45a8d19097f2e930bc
I20260809 06:26:39.958762 24658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:26:39.976331 24658 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:26:39.977634 24658 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:26:39.987157 24658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e: Generated new TSK 0
I20260809 06:26:39.988032 24658 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:26:40.058911 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44407
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:40.395136 24680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:40.395393 24680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:40.395537 24680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:40.402393 24680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:40.402495 24680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:40.402657 24680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:26:40.411739 24680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:40.412900 24680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:40.414180 24680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:40.427709 24692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:40.427953 24687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:40.427743 24688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:40.428838 24680 server_base.cc:1034] running on GCE node
I20260809 06:26:40.429481 24680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:40.430358 24680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:40.431584 24680 hybrid_clock.cc:648] HybridClock initialized: now 1786256800431525 us; error 54 us; skew 500 ppm
I20260809 06:26:40.431918 24680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:40 dist-test-slave-mjjr krb5kdc[24522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256800, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:40.435045 24680 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:26:40.436466 24680 webserver.cc:460] Webserver started at http://127.8.131.1:38347/ using document root <none> and password file <none>
I20260809 06:26:40.437126 24680 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:40.437304 24680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:40.437608 24680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:40.440691 24680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "0395ecfed6c04019aed962055b3e303a"
format_stamp: "Formatted at 2026-08-09 06:26:40 on dist-test-slave-mjjr"
server_key: "74276382fcc5901aded4a67491a80f69"
server_key_iv: "5fbfdc4ee3ca974aa9c30e5589767bca"
server_key_version: "encryptionkey@0"
I20260809 06:26:40.441511 24680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "0395ecfed6c04019aed962055b3e303a"
format_stamp: "Formatted at 2026-08-09 06:26:40 on dist-test-slave-mjjr"
server_key: "74276382fcc5901aded4a67491a80f69"
server_key_iv: "5fbfdc4ee3ca974aa9c30e5589767bca"
server_key_version: "encryptionkey@0"
I20260809 06:26:40.446689 24680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:26:40.449990 24708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:40.451234 24680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:26:40.451406 24680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "0395ecfed6c04019aed962055b3e303a"
format_stamp: "Formatted at 2026-08-09 06:26:40 on dist-test-slave-mjjr"
server_key: "74276382fcc5901aded4a67491a80f69"
server_key_iv: "5fbfdc4ee3ca974aa9c30e5589767bca"
server_key_version: "encryptionkey@0"
I20260809 06:26:40.451597 24680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:40.476305 24680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:40.480227 24680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:40.480525 24680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:40.481478 24680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:40.482813 24680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:40.482923 24680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:40.483062 24680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:40.483136 24680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:40.516716 24680 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:39075
I20260809 06:26:40.516769 24877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:39075 every 8 connection(s)
I20260809 06:26:40.518067 24680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:26:40.525446 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 24680
I20260809 06:26:40.525686 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:26:40.526526 8716 external_mini_cluster.cc:1428] Setting key 5e0d49a8d6efba30f4fe8c5ebb822543
Aug 09 06:26:40 dist-test-slave-mjjr krb5kdc[24522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256800, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:26:40.540014 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:40.520887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:56111 (local address 127.8.131.62:44407)
0809 06:26:40.521342 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:40.521346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:40.522297 (+ 951us) server_negotiation.cc:407] Connection header received
0809 06:26:40.524105 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:40.524110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:40.524179 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:40.524265 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:40.525752 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:40.526917 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:40.528831 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:40.529520 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:40.529691 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:40.533276 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:40.533284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:40.533292 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:40.533336 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:40.536953 (+ 3617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:40.537661 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:40.537668 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:40.537674 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:40.537848 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:40.538243 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:40.538250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:40.538255 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:40.538648 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:40.538799 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:40.539462 (+ 663us) server_negotiation.cc:299] Negotiation successful
0809 06:26:40.539740 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":219,"threads_started":1}
I20260809 06:26:40.541333 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:40.521400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:44407 (local address 127.8.131.1:56111)
0809 06:26:40.522134 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:40.522179 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:40.523844 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:40.524470 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:40.524477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:40.524971 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:40.525559 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:40.525583 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:40.527079 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:40.527083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:40.528682 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:40.528694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:40.529773 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:40.529778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:40.529966 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:40.530640 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:40.530662 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:40.533034 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:40.537085 (+ 4051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:40.537092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:40.537103 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:40.537533 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:40.537950 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:40.537954 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:40.537956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:40.538132 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:40.538770 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:40.538776 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:40.539082 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 06:26:40.539301 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 06:26:40.539886 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":270,"threads_started":1}
I20260809 06:26:40.542834 24878 heartbeater.cc:347] Connected to a master server at 127.8.131.62:44407
I20260809 06:26:40.543138 24878 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:40.543915 24878 heartbeater.cc:511] Master 127.8.131.62:44407 requested a full tablet report, sending...
I20260809 06:26:40.545984 24588 ts_manager.cc:194] Registered new tserver with Master: 0395ecfed6c04019aed962055b3e303a (127.8.131.1:39075)
I20260809 06:26:40.547410 24588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:56111
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:26:40.587397 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44407
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:40.917469 24886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:40.917757 24886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:40.917939 24886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:40.924993 24886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:40.925123 24886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:40.925303 24886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:26:40.934625 24886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:40.936107 24886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:40.937469 24886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:40.950333 24896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:40.950412 24898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:40.950378 24893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:40.951221 24886 server_base.cc:1034] running on GCE node
I20260809 06:26:40.951821 24886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:40.952701 24886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:40.953907 24886 hybrid_clock.cc:648] HybridClock initialized: now 1786256800953843 us; error 49 us; skew 500 ppm
I20260809 06:26:40.954212 24886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:40 dist-test-slave-mjjr krb5kdc[24522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256800, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:40.957410 24886 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:26:40.958766 24886 webserver.cc:460] Webserver started at http://127.8.131.2:38983/ using document root <none> and password file <none>
I20260809 06:26:40.959477 24886 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:40.959616 24886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:40.960026 24886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:40.963075 24886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "88c3862681af405398a1a8d0995d062b"
format_stamp: "Formatted at 2026-08-09 06:26:40 on dist-test-slave-mjjr"
server_key: "3d0c68fee0962bf07912f2e74b5565ed"
server_key_iv: "6c4db40fd9205f2ceb5bc3a500d2120e"
server_key_version: "encryptionkey@0"
I20260809 06:26:40.963899 24886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "88c3862681af405398a1a8d0995d062b"
format_stamp: "Formatted at 2026-08-09 06:26:40 on dist-test-slave-mjjr"
server_key: "3d0c68fee0962bf07912f2e74b5565ed"
server_key_iv: "6c4db40fd9205f2ceb5bc3a500d2120e"
server_key_version: "encryptionkey@0"
I20260809 06:26:40.968822 24886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:26:40.972111 24907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:40.973498 24886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 06:26:40.973675 24886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "88c3862681af405398a1a8d0995d062b"
format_stamp: "Formatted at 2026-08-09 06:26:40 on dist-test-slave-mjjr"
server_key: "3d0c68fee0962bf07912f2e74b5565ed"
server_key_iv: "6c4db40fd9205f2ceb5bc3a500d2120e"
server_key_version: "encryptionkey@0"
I20260809 06:26:40.973901 24886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:41.004896 24886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:41.008814 24886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:41.009120 24886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:41.010103 24886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:41.011400 24886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:41.011503 24886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:41.011664 24886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:41.011746 24886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 06:26:41.044623 24886 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:34217
I20260809 06:26:41.044670 25051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:34217 every 8 connection(s)
I20260809 06:26:41.045910 24886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:26:41.054816 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 24886
I20260809 06:26:41.055035 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:26:41.055516 8716 external_mini_cluster.cc:1428] Setting key 172642d4cabc01da5338d8cd617f4fc7
Aug 09 06:26:41 dist-test-slave-mjjr krb5kdc[24522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256800, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:26:41.067167 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.048473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:53961 (local address 127.8.131.62:44407)
0809 06:26:41.048903 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.048908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.049821 (+ 913us) server_negotiation.cc:407] Connection header received
0809 06:26:41.051520 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.051524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.051591 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.051757 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:41.053160 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.054017 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.055822 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.056426 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.056555 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.060232 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:41.060241 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:41.060248 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:41.060292 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:41.063967 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.064629 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.064637 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.064643 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.064806 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.065198 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.065206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.065212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.065603 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:41.065774 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.066392 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.066660 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":212,"threads_started":1}
I20260809 06:26:41.067804 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.049003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:44407 (local address 127.8.131.2:53961)
0809 06:26:41.049664 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:41.049701 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:41.051270 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:41.051929 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:41.051936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:41.052423 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:41.052989 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.053005 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.054149 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.054152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.055683 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.055697 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.056626 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.056650 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.056945 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.057715 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:41.057746 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:41.060035 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:41.064091 (+ 4056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:41.064098 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:41.064110 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:41.064517 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:41.064907 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:41.064910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:41.064912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:41.065095 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:41.065717 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:41.065722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:41.066031 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 06:26:41.066265 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 06:26:41.066655 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":276,"threads_started":1}
I20260809 06:26:41.069233 25052 heartbeater.cc:347] Connected to a master server at 127.8.131.62:44407
I20260809 06:26:41.069531 25052 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:41.070236 25052 heartbeater.cc:511] Master 127.8.131.62:44407 requested a full tablet report, sending...
I20260809 06:26:41.071749 24588 ts_manager.cc:194] Registered new tserver with Master: 88c3862681af405398a1a8d0995d062b (127.8.131.2:34217)
I20260809 06:26:41.073046 24588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:53961
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:26:41.112526 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44407
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:26:41.439147 25060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:26:41.439411 25060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:26:41.439563 25060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:26:41.446846 25060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:26:41.446965 25060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:26:41.447136 25060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:26:41.456555 25060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:26:41.457759 25060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:26:41.459030 25060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:26:41.472195 25070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:41.472295 25066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:26:41.472231 25068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:26:41.473522 25060 server_base.cc:1034] running on GCE node
I20260809 06:26:41.474076 25060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:26:41.474902 25060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:26:41.476085 25060 hybrid_clock.cc:648] HybridClock initialized: now 1786256801476028 us; error 59 us; skew 500 ppm
I20260809 06:26:41.476383 25060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:26:41 dist-test-slave-mjjr krb5kdc[24522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:26:41.479933 25060 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:26:41.481365 25060 webserver.cc:460] Webserver started at http://127.8.131.3:34763/ using document root <none> and password file <none>
I20260809 06:26:41.482017 25060 fs_manager.cc:359] Metadata directory not provided
I20260809 06:26:41.482160 25060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:26:41.482455 25060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:26:41.485344 25060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "aca9a5b103c14046be30d550edbf960d"
format_stamp: "Formatted at 2026-08-09 06:26:41 on dist-test-slave-mjjr"
server_key: "6f507b985c385c9d1141303df764f7fd"
server_key_iv: "5a04ebaa044823f76a5641f500a88784"
server_key_version: "encryptionkey@0"
I20260809 06:26:41.486124 25060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "aca9a5b103c14046be30d550edbf960d"
format_stamp: "Formatted at 2026-08-09 06:26:41 on dist-test-slave-mjjr"
server_key: "6f507b985c385c9d1141303df764f7fd"
server_key_iv: "5a04ebaa044823f76a5641f500a88784"
server_key_version: "encryptionkey@0"
I20260809 06:26:41.491000 25060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 06:26:41.494743 25081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:41.496068 25060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 06:26:41.496236 25060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "aca9a5b103c14046be30d550edbf960d"
format_stamp: "Formatted at 2026-08-09 06:26:41 on dist-test-slave-mjjr"
server_key: "6f507b985c385c9d1141303df764f7fd"
server_key_iv: "5a04ebaa044823f76a5641f500a88784"
server_key_version: "encryptionkey@0"
I20260809 06:26:41.496452 25060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:26:41.506727 25060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:26:41.510901 25060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:26:41.511204 25060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:26:41.512133 25060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:26:41.513429 25060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:26:41.513540 25060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:41.513682 25060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:26:41.513759 25060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:26:41.547422 25060 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:40131
I20260809 06:26:41.547508 25232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:40131 every 8 connection(s)
I20260809 06:26:41.548712 25060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:26:41.550262 24878 heartbeater.cc:503] Master 127.8.131.62:44407 was elected leader, sending a full tablet report...
I20260809 06:26:41.556066 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 25060
I20260809 06:26:41.556289 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:26:41.556731 8716 external_mini_cluster.cc:1428] Setting key 457a51b2761276b73b6b1a17dd4eddd7
Aug 09 06:26:41 dist-test-slave-mjjr krb5kdc[24522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:26:41.567574 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.551151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:60259 (local address 127.8.131.62:44407)
0809 06:26:41.551316 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.551321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.552576 (+ 1255us) server_negotiation.cc:407] Connection header received
0809 06:26:41.554387 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.554391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.554472 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.554564 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:41.556141 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.556971 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.558631 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.559074 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.559200 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.562651 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:41.562656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:41.562659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:41.562687 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:41.565108 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.565751 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.565755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.565757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.565867 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.566217 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.566220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.566222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.566493 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:41.566602 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.567162 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.567318 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 06:26:41.568454 25236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.551683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:44407 (local address 127.8.131.3:60259)
0809 06:26:41.552440 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:41.552480 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:41.554061 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:41.554747 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:41.554754 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:41.555324 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:26:41.555987 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.556005 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.557128 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.557133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.558479 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.558496 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.559299 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.559308 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.559558 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.560232 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:26:41.560253 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:26:41.562371 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:26:41.565239 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:41.565245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:41.565255 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:41.565648 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:41.565951 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:26:41.565954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:26:41.565956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:26:41.566126 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:26:41.566592 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:26:41.566598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:26:41.566851 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 06:26:41.567071 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 06:26:41.567402 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":324,"threads_started":1}
I20260809 06:26:41.569830 25233 heartbeater.cc:347] Connected to a master server at 127.8.131.62:44407
I20260809 06:26:41.570127 25233 heartbeater.cc:464] Registering TS with master...
I20260809 06:26:41.570782 25233 heartbeater.cc:511] Master 127.8.131.62:44407 requested a full tablet report, sending...
I20260809 06:26:41.572132 24588 ts_manager.cc:194] Registered new tserver with Master: aca9a5b103c14046be30d550edbf960d (127.8.131.3:40131)
I20260809 06:26:41.572846 24588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:60259
I20260809 06:26:41.583698 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:26:41.600780 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.588246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.8.131.62:44407)
0809 06:26:41.588434 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.588438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.588929 (+ 491us) server_negotiation.cc:407] Connection header received
0809 06:26:41.589109 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.589115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.589195 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.589290 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:41.590703 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.591465 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.592804 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.593254 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.593382 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.595594 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:41.595599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:41.595603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:41.595677 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:41.598047 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.598718 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.598723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.598727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.598870 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.599228 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.599233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.599238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.599623 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:41.599921 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.600226 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.600446 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 06:26:41.604382 24588 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:26:41.611096 24588 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:26:41.633947 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.622741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56868 (local address 127.8.131.3:40131)
0809 06:26:41.625240 (+ 2499us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.625248 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.625277 (+ 29us) server_negotiation.cc:407] Connection header received
0809 06:26:41.625358 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.625364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.625571 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.626576 (+ 1005us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:41.626901 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.628414 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.631342 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.632785 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.632940 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.633199 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.633445 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.633608 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2328,"thread_start_us":2157,"threads_started":1}
I20260809 06:26:41.634079 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.622553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:40131 (local address 127.0.0.1:56868)
0809 06:26:41.623465 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:41.623507 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:41.623721 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:41.625807 (+ 2086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:41.625814 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:41.625853 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:41.626332 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.626356 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.628571 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.628576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.631171 (+ 2595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.631189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.633008 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.633013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.633198 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.633263 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 06:26:41.633404 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 06:26:41.633607 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":39,"thread_start_us":1425,"threads_started":1}
I20260809 06:26:41.635200 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.623221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.8.131.2:34217)
0809 06:26:41.624281 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.624290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.624835 (+ 545us) server_negotiation.cc:407] Connection header received
0809 06:26:41.625172 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.625181 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.625429 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.625613 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:41.626656 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.628202 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.631599 (+ 3397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.633010 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.633135 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.633312 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.634719 (+ 1407us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.634895 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919,"thread_start_us":836,"threads_started":1}
I20260809 06:26:41.635418 25253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.622677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:34217 (local address 127.0.0.1:35926)
0809 06:26:41.624718 (+ 2041us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:41.624736 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:41.624947 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:41.625928 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:41.625934 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:41.625958 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:41.626433 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.626451 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.628413 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.628419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.631431 (+ 3012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.631444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.633375 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.633384 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.633569 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.633603 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:26:41.633703 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 06:26:41.633838 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1920,"mutex_wait_us":635,"spinlock_wait_cycles":9088,"thread_start_us":1907,"threads_started":1}
I20260809 06:26:41.637765 25257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.624898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:39075 (local address 127.0.0.1:37866)
0809 06:26:41.629423 (+ 4525us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:41.629442 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:41.629557 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:41.630716 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:41.630722 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:41.630750 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:41.631194 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.631204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.632662 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.632668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.635086 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.635100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.636987 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.636994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.637175 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.637211 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 06:26:41.637310 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 06:26:41.637457 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4384,"thread_start_us":4221,"threads_started":1}
I20260809 06:26:41.637833 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.625531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37866 (local address 127.8.131.1:39075)
0809 06:26:41.629856 (+ 4325us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.629864 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.629893 (+ 29us) server_negotiation.cc:407] Connection header received
0809 06:26:41.629998 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.630005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.630237 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.630470 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:41.631410 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.632518 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.635283 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.636465 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.636609 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.636844 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.637383 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.637535 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4146,"thread_start_us":3969,"threads_started":1}
I20260809 06:26:41.638027 25140 tablet_service.cc:1467] Processing CreateTablet for tablet 78c704a03c7c4200a17809f2fa4451b0 (DEFAULT_TABLE table=test-table [id=c65f9fd66dd14f4db33ae5673baac734]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:41.639103 24971 tablet_service.cc:1467] Processing CreateTablet for tablet 78c704a03c7c4200a17809f2fa4451b0 (DEFAULT_TABLE table=test-table [id=c65f9fd66dd14f4db33ae5673baac734]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:41.639766 25140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78c704a03c7c4200a17809f2fa4451b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:41.640487 24971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78c704a03c7c4200a17809f2fa4451b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:41.641644 24785 tablet_service.cc:1467] Processing CreateTablet for tablet 78c704a03c7c4200a17809f2fa4451b0 (DEFAULT_TABLE table=test-table [id=c65f9fd66dd14f4db33ae5673baac734]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:26:41.643090 24785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78c704a03c7c4200a17809f2fa4451b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:26:41.650956 25261 tablet_bootstrap.cc:492] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: Bootstrap starting.
I20260809 06:26:41.654101 25261 tablet_bootstrap.cc:654] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:41.654960 25262 tablet_bootstrap.cc:492] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: Bootstrap starting.
I20260809 06:26:41.656787 25261 log.cc:824] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:41.657651 25263 tablet_bootstrap.cc:492] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: Bootstrap starting.
I20260809 06:26:41.658582 25262 tablet_bootstrap.cc:654] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:41.659958 25262 log.cc:824] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:41.661177 25263 tablet_bootstrap.cc:654] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: Neither blocks nor log segments found. Creating new log.
I20260809 06:26:41.661450 25261 tablet_bootstrap.cc:492] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: No bootstrap required, opened a new log
I20260809 06:26:41.661698 25261 ts_tablet_manager.cc:1397] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260809 06:26:41.662424 25263 log.cc:824] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: Log is configured to *not* fsync() on all Append() calls
I20260809 06:26:41.663316 25262 tablet_bootstrap.cc:492] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: No bootstrap required, opened a new log
I20260809 06:26:41.663558 25262 ts_tablet_manager.cc:1397] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260809 06:26:41.666028 25263 tablet_bootstrap.cc:492] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: No bootstrap required, opened a new log
I20260809 06:26:41.666342 25263 ts_tablet_manager.cc:1397] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 06:26:41.669860 25261 raft_consensus.cc:348] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } }
I20260809 06:26:41.670246 25261 raft_consensus.cc:374] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:41.670348 25261 raft_consensus.cc:729] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca9a5b103c14046be30d550edbf960d, State: Initialized, Role: FOLLOWER
I20260809 06:26:41.670964 25261 consensus_queue.cc:260] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } }
I20260809 06:26:41.672060 25233 heartbeater.cc:503] Master 127.8.131.62:44407 was elected leader, sending a full tablet report...
I20260809 06:26:41.673779 25261 ts_tablet_manager.cc:1428] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: Time spent starting tablet: real 0.012s user 0.003s sys 0.009s
I20260809 06:26:41.674110 25262 raft_consensus.cc:348] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } }
I20260809 06:26:41.674551 25262 raft_consensus.cc:374] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:41.674649 25262 raft_consensus.cc:729] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c3862681af405398a1a8d0995d062b, State: Initialized, Role: FOLLOWER
I20260809 06:26:41.675364 25262 consensus_queue.cc:260] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } }
I20260809 06:26:41.677013 25052 heartbeater.cc:503] Master 127.8.131.62:44407 was elected leader, sending a full tablet report...
I20260809 06:26:41.677120 25262 ts_tablet_manager.cc:1428] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260809 06:26:41.679787 25263 raft_consensus.cc:348] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } }
I20260809 06:26:41.680217 25263 raft_consensus.cc:374] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:26:41.680322 25263 raft_consensus.cc:729] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0395ecfed6c04019aed962055b3e303a, State: Initialized, Role: FOLLOWER
I20260809 06:26:41.681037 25263 consensus_queue.cc:260] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } }
I20260809 06:26:41.682677 25263 ts_tablet_manager.cc:1428] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: Time spent starting tablet: real 0.016s user 0.012s sys 0.000s
I20260809 06:26:41.704814 25272 raft_consensus.cc:482] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:26:41.705137 25272 raft_consensus.cc:504] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } }
I20260809 06:26:41.706660 25272 leader_election.cc:288] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aca9a5b103c14046be30d550edbf960d (127.8.131.3:40131), 0395ecfed6c04019aed962055b3e303a (127.8.131.1:39075)
I20260809 06:26:41.713085 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.707094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:58099 (local address 127.8.131.3:40131)
0809 06:26:41.707336 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.707345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.707812 (+ 467us) server_negotiation.cc:407] Connection header received
0809 06:26:41.708027 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.708035 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.708146 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.708246 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:41.708788 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.709523 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.711311 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.712153 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.712299 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.712417 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.712697 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.712813 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:26:41.713254 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.707025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:40131 (local address 127.8.131.2:58099)
0809 06:26:41.707680 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:41.707702 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:41.707893 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:41.708274 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:41.708286 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:41.708317 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:41.708643 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.708652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.709655 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.709662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.711158 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.711170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.712364 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.712370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.712545 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.712568 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 06:26:41.712670 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 06:26:41.712787 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":1,"thread_start_us":353,"threads_started":1}
I20260809 06:26:41.713701 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.707937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:36693 (local address 127.8.131.1:39075)
0809 06:26:41.708167 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.708174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.708574 (+ 400us) server_negotiation.cc:407] Connection header received
0809 06:26:41.708765 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.708775 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.708831 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.708923 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:26:41.709460 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.710123 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.711962 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.712749 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.712903 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.713004 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.713402 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.713536 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:26:41.714249 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.707877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:39075 (local address 127.8.131.2:36693)
0809 06:26:41.708476 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0809 06:26:41.708492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:26:41.708634 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:26:41.708943 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:26:41.708947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:26:41.708971 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:26:41.709296 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.709304 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.710279 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.710284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.711816 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:26:41.711828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.712980 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.712987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:26:41.713197 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.713250 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 06:26:41.713890 (+ 640us) client_negotiation.cc:240] Negotiation successful
0809 06:26:41.713980 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":27,"thread_start_us":262,"threads_started":1}
I20260809 06:26:41.714221 25166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78c704a03c7c4200a17809f2fa4451b0" candidate_uuid: "88c3862681af405398a1a8d0995d062b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca9a5b103c14046be30d550edbf960d" is_pre_election: true
I20260809 06:26:41.714720 25166 raft_consensus.cc:2454] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88c3862681af405398a1a8d0995d062b in term 0.
I20260809 06:26:41.715245 24812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78c704a03c7c4200a17809f2fa4451b0" candidate_uuid: "88c3862681af405398a1a8d0995d062b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0395ecfed6c04019aed962055b3e303a" is_pre_election: true
I20260809 06:26:41.715431 24910 leader_election.cc:302] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88c3862681af405398a1a8d0995d062b, aca9a5b103c14046be30d550edbf960d; no voters:
I20260809 06:26:41.715765 24812 raft_consensus.cc:2454] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88c3862681af405398a1a8d0995d062b in term 0.
I20260809 06:26:41.715854 25272 raft_consensus.cc:2781] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:26:41.716013 25272 raft_consensus.cc:482] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:26:41.716127 25272 raft_consensus.cc:3037] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:41.718313 25272 raft_consensus.cc:504] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } }
I20260809 06:26:41.718834 25272 leader_election.cc:288] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [CANDIDATE]: Term 1 election: Requested vote from peers aca9a5b103c14046be30d550edbf960d (127.8.131.3:40131), 0395ecfed6c04019aed962055b3e303a (127.8.131.1:39075)
I20260809 06:26:41.719264 25166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78c704a03c7c4200a17809f2fa4451b0" candidate_uuid: "88c3862681af405398a1a8d0995d062b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca9a5b103c14046be30d550edbf960d"
I20260809 06:26:41.719347 24812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78c704a03c7c4200a17809f2fa4451b0" candidate_uuid: "88c3862681af405398a1a8d0995d062b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0395ecfed6c04019aed962055b3e303a"
I20260809 06:26:41.719447 25166 raft_consensus.cc:3037] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:41.719563 24812 raft_consensus.cc:3037] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:26:41.721686 25166 raft_consensus.cc:2454] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88c3862681af405398a1a8d0995d062b in term 1.
I20260809 06:26:41.721822 24812 raft_consensus.cc:2454] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88c3862681af405398a1a8d0995d062b in term 1.
I20260809 06:26:41.722054 24910 leader_election.cc:302] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88c3862681af405398a1a8d0995d062b, aca9a5b103c14046be30d550edbf960d; no voters:
I20260809 06:26:41.722373 25272 raft_consensus.cc:2781] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:26:41.722843 25272 raft_consensus.cc:686] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 1 LEADER]: Becoming Leader. State: Replica: 88c3862681af405398a1a8d0995d062b, State: Running, Role: LEADER
I20260809 06:26:41.723356 25272 consensus_queue.cc:237] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } }
I20260809 06:26:41.727604 24588 catalog_manager.cc:5495] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b reported cstate change: term changed from 0 to 1, leader changed from <none> to 88c3862681af405398a1a8d0995d062b (127.8.131.2). New cstate: current_term: 1 leader_uuid: "88c3862681af405398a1a8d0995d062b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88c3862681af405398a1a8d0995d062b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260809 06:26:41.759050 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.753634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.8.131.2:34217)
0809 06:26:41.753851 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.753857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.753965 (+ 108us) server_negotiation.cc:407] Connection header received
0809 06:26:41.754199 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.754203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.754274 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.754444 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:26:41.754826 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.755614 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.757006 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.757419 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.757548 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.758009 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:26:41.758481 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:26:41.758631 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.758691 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.758770 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:26:41.766940 25166 raft_consensus.cc:1261] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [term 1 FOLLOWER]: Refusing update from remote peer 88c3862681af405398a1a8d0995d062b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:41.766944 24812 raft_consensus.cc:1261] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [term 1 FOLLOWER]: Refusing update from remote peer 88c3862681af405398a1a8d0995d062b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:26:41.767889 25278 consensus_queue.cc:1035] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0395ecfed6c04019aed962055b3e303a" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:26:41.768236 25272 consensus_queue.cc:1035] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [LEADER]: Connected to new peer: Peer: permanent_uuid: "aca9a5b103c14046be30d550edbf960d" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 40131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 06:26:41.770529 24879 tablet.cc:2369] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:26:41.781275 25283 mvcc.cc:205] Tried to move back new op lower bound from 7316507860028055552 to 7316507859863035904. Current Snapshot: MvccSnapshot[applied={T|T < 7316507860028055552}]
I20260809 06:26:41.791900 24588 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:41.792169 24588 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:26:41.795394 24588 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63b244fba13e4fff92d2bcb69a16c87e: Sending DeleteTablet for 3 replicas of tablet 78c704a03c7c4200a17809f2fa4451b0
I20260809 06:26:41.796483 25140 tablet_service.cc:1514] Processing DeleteTablet for tablet 78c704a03c7c4200a17809f2fa4451b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:41 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:56868
I20260809 06:26:41.796654 24971 tablet_service.cc:1514] Processing DeleteTablet for tablet 78c704a03c7c4200a17809f2fa4451b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:41 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:35926
I20260809 06:26:41.797849 24785 tablet_service.cc:1514] Processing DeleteTablet for tablet 78c704a03c7c4200a17809f2fa4451b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:26:41 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:37866
I20260809 06:26:41.799419 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 24680
I20260809 06:26:41.812306 25297 tablet_replica.cc:330] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: stopping tablet replica
I20260809 06:26:41.812811 25293 tablet_replica.cc:330] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: stopping tablet replica
I20260809 06:26:41.813225 25293 raft_consensus.cc:2229] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:41.813512 25297 raft_consensus.cc:2229] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:26:41.813776 25293 raft_consensus.cc:2258] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:41.814489 25297 raft_consensus.cc:2258] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:41.814606 25296 tablet_replica.cc:330] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: stopping tablet replica
I20260809 06:26:41.814990 25296 raft_consensus.cc:2229] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:26:41.815943 25296 raft_consensus.cc:2258] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:26:41.818463 25293 ts_tablet_manager.cc:1905] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:41.818732 25297 ts_tablet_manager.cc:1905] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:26:41.823415 25296 ts_tablet_manager.cc:1905] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 06:26:41 dist-test-slave-mjjr krb5kdc[24522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:26:41.825907 25297 ts_tablet_manager.cc:1918] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:41.826071 25297 log.cc:1196] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/78c704a03c7c4200a17809f2fa4451b0
I20260809 06:26:41.826620 25297 ts_tablet_manager.cc:1939] T 78c704a03c7c4200a17809f2fa4451b0 P 0395ecfed6c04019aed962055b3e303a: Deleting consensus metadata
I20260809 06:26:41.828609 24569 catalog_manager.cc:4841] TS 0395ecfed6c04019aed962055b3e303a (127.8.131.1:39075): tablet 78c704a03c7c4200a17809f2fa4451b0 (table test-table [id=c65f9fd66dd14f4db33ae5673baac734]) successfully deleted
I20260809 06:26:41.832307 25293 ts_tablet_manager.cc:1918] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:41.832509 25293 log.cc:1196] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/78c704a03c7c4200a17809f2fa4451b0
I20260809 06:26:41.833081 25293 ts_tablet_manager.cc:1939] T 78c704a03c7c4200a17809f2fa4451b0 P aca9a5b103c14046be30d550edbf960d: Deleting consensus metadata
I20260809 06:26:41.833093 25296 ts_tablet_manager.cc:1918] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:26:41.833250 25296 log.cc:1196] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/78c704a03c7c4200a17809f2fa4451b0
I20260809 06:26:41.833801 25296 ts_tablet_manager.cc:1939] T 78c704a03c7c4200a17809f2fa4451b0 P 88c3862681af405398a1a8d0995d062b: Deleting consensus metadata
I20260809 06:26:41.834965 24569 catalog_manager.cc:4841] TS aca9a5b103c14046be30d550edbf960d (127.8.131.3:40131): tablet 78c704a03c7c4200a17809f2fa4451b0 (table test-table [id=c65f9fd66dd14f4db33ae5673baac734]) successfully deleted
I20260809 06:26:41.835072 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.803479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37870 (local address 127.8.131.1:39075)
0809 06:26:41.803719 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.803728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.814011 (+ 10283us) server_negotiation.cc:407] Connection header received
0809 06:26:41.814909 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.814916 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.815031 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.815115 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:41.817054 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.818272 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.820310 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.820993 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.821099 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.826896 (+ 5797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:41.826911 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:41.826929 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:41.826978 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:41.831016 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.831968 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.831978 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.831982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.832149 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.832849 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.832854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.832857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.833492 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:41.833792 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.834463 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.834701 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 06:26:41.835350 24571 catalog_manager.cc:4841] TS 88c3862681af405398a1a8d0995d062b (127.8.131.2:34217): tablet 78c704a03c7c4200a17809f2fa4451b0 (table test-table [id=c65f9fd66dd14f4db33ae5673baac734]) successfully deleted
I20260809 06:26:41.836918 24868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:41.931964 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 24680
I20260809 06:26:41.962494 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 24886
Aug 09 06:26:41 dist-test-slave-mjjr krb5kdc[24522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:26:41.981139 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:41.963382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.8.131.2:34217)
0809 06:26:41.963570 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:41.963574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:41.963686 (+ 112us) server_negotiation.cc:407] Connection header received
0809 06:26:41.963866 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:41.963870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:41.963933 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:41.964019 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:41.965646 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.966447 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.968373 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:41.968805 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:41.968912 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:41.973945 (+ 5033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:41.973965 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:41.973992 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:41.974048 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:41.978158 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.978905 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.978921 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.978924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.979108 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:41.979460 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:41.979465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:41.979469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:41.979979 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:41.980316 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:41.980503 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 06:26:41.980721 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 06:26:41.981850 25047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:42.092868 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 24886
I20260809 06:26:42.117107 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 25060
Aug 09 06:26:42 dist-test-slave-mjjr krb5kdc[24522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:26:42.130904 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:26:42.117851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56870 (local address 127.8.131.3:40131)
0809 06:26:42.118039 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 06:26:42.118044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:26:42.118064 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:26:42.118215 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:26:42.118219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:26:42.118294 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:26:42.118390 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:26:42.119595 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:42.120352 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:42.121507 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:26:42.121883 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:26:42.121972 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:26:42.126027 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:26:42.126040 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:26:42.126055 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:26:42.126084 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:26:42.128660 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:42.129269 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:42.129276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:42.129278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:42.129419 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:26:42.129712 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:26:42.129716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:26:42.129717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:26:42.130041 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:26:42.130251 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 06:26:42.130489 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 06:26:42.130650 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 06:26:42.131582 25227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:42.219383 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 25060
I20260809 06:26:42.243005 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 24542
I20260809 06:26:42.244277 24636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:26:42.357270 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 24542
2026-08-09T06:26:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4476 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:26:42 dist-test-slave-mjjr krb5kdc[25309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:26:42 dist-test-slave-mjjr krb5kdc[25309](info): set up 2 sockets
Aug 09 06:26:42 dist-test-slave-mjjr krb5kdc[25309](info): commencing operation
krb5kdc: starting...
W20260809 06:26:43.708011 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:26:43 dist-test-slave-mjjr krb5kdc[25309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:26:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:26:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.131.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.131.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/rangeradmin_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/rangeradmin_127.8.131.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.131.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.131.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/rangerlookup_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/rangerlookup_127.8.131.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.131.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.131.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/HTTP_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/HTTP_127.8.131.20@KRBTEST.COM.keytab.
I20260809 06:26:44.110281 8716 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskn2CWnW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/postgres -l logfile start
2026-08-09 06:26:46.512 UTC [25356] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 06:26:46.512 UTC [25356] LOG: listening on IPv4 address "127.8.131.20", port 47247
2026-08-09 06:26:46.516 UTC [25356] LOG: listening on Unix socket "/tmp/.s.PGSQL.47247"
2026-08-09 06:26:46.540 UTC [25361] LOG: database system was shut down at 2026-08-09 06:26:45 UTC
2026-08-09 06:26:46.544 UTC [25356] LOG: database system is ready to accept connections
I20260809 06:26:47.704252 8716 mini_postgres.cc:96] Postgres bound to 47247
2026-08-09 06:26:47.728 UTC [25372] FATAL: database "slave" does not exist
127.8.131.20:47247 - accepting connections
I20260809 06:26:47.729396 8716 mini_ranger.cc:161] Starting Ranger...
I20260809 06:26:47.758718 8716 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 06:26:48.198887 8716 mini_ranger.cc:87] Created ranger Postgres database
I20260809 06:26:48.199039 8716 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-admin
2026-08-09 06:26:48,531 [I] DB FLAVOR :POSTGRES
2026-08-09 06:26:48,532 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 06:26:48,532 [I] Checking connection..
2026-08-09 06:26:48,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 06:26:48,799 [I] Checking connection passed.
2026-08-09 06:26:48,799 [I] --------- Verifying version history table ---------
2026-08-09 06:26:48,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 06:26:49,055 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 06:26:49,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 06:26:49,302 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 06:26:49,302 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 06:26:49,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 06:26:49.551 UTC [25480] WARNING: there is no transaction in progress
2026-08-09 06:26:49,558 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 06:26:49,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 06:26:49,806 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 06:26:49,806 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 06:26:49,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 06:26:50,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 06:26:50,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mjjr.c.gcp-upstream.internal','N') ;"
2026-08-09 06:26:50,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 06:26:50,812 [I] Table x_portal_user does not exist in database ranger
2026-08-09 06:26:50,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 06:26:51,075 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 06:26:51,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 06:26:51,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 06:26:51,590 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 06:26:51,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 06:26:52.533 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.549 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.564 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.579 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.594 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.609 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.661 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.667 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.680 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.691 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.702 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.715 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.724 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.730 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.737 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:52.742 UTC [25647] WARNING: there is no transaction in progress
2026-08-09 06:26:53,326 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 06:26:53,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 06:26:53,586 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 06:26:53,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 06:26:53,839 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 06:26:53,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 06:26:54,116 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 06:26:54,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 06:26:54,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 06:26:54,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mjjr.c.gcp-upstream.internal';"
2026-08-09 06:26:54,871 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 06:26:54,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 06:26:55,141 [I] DB_PATCHES have already been applied
I20260809 06:26:55.147445 8716 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-admin:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskn2CWnW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 06:26:55.147575 8716 mini_ranger.cc:193] Using host: 127.8.131.20
I20260809 06:26:55.167544 8716 mini_ranger.cc:227] Ranger admin URL: http://127.8.131.20:49639
Aug 09, 2026 6:26:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 6:26:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 6:26:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 6:26:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 6:26:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 6:26:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 6:26:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 06:26:56.551487 8716 mini_ranger.cc:160] Time spent starting Ranger: real 8.822s user 0.012s sys 0.042s
Aug 09, 2026 6:27:16 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 6:27:16 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 6:27:16 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 6:27:16 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 6:27:17 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 06:27:18.215524 8716 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.131.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.131.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/rangerkms_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/rangerkms_127.8.131.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.131.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.131.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.131.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/HTTP_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.131.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/HTTP_127.8.131.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 06:27:18 dist-test-slave-mjjr krb5kdc[25309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256838, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 06:27:18.431859 8716 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 06:27:18.458885 8716 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 06:27:19.240378 8716 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 06:27:19.240576 8716 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-kms
I20260809 06:27:19.240777 8716 mini_ranger_kms.cc:227] Using postgres at 127.8.131.20:47247
2026-08-09 06:27:19,733 [I] DB FLAVOR :POSTGRES
2026-08-09 06:27:19,733 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 06:27:19,733 [I] Checking connection
2026-08-09 06:27:19,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 06:27:19,983 [I] connection success
2026-08-09 06:27:19,983 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 06:27:19,983 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 06:27:19,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 06:27:20,218 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 06:27:20,219 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 06:27:20,219 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 06:27:20,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:47247/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 06:27:20,510 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 06:27:20.952906 8716 mini_ranger_kms.cc:310] Created kms service
I20260809 06:27:21.141304 8716 mini_ranger_kms.cc:326] Created kudu user
I20260809 06:27:21.185606 8716 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 06:27:21 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256838, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:27:21.599875 8716 mini_ranger_kms.cc:384] Added ranger policy
I20260809 06:27:21.601320 8716 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-kms:/tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 06:27:21.601413 8716 mini_ranger_kms.cc:240] Using host: 127.8.131.20
I20260809 06:27:21.629973 8716 mini_ranger_kms.cc:276] Ranger KMS PID: 25900
I20260809 06:27:21.630170 8716 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.131.20:43459
Aug 09, 2026 6:27:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 6:27:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 6:27:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 6:27:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 6:27:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 6:27:23 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 06:27:24 dist-test-slave-mjjr krb5kdc[25309](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786256844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.131.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:27:24.443490 8716 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.012s user 0.101s sys 0.428s
I20260809 06:27:24.444097 8716 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 06:27:24.444329 8716 mini_ranger_kms.cc:401] 127.8.131.20:43459/kms/v1/keys
Aug 09 06:27:24 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.131.20@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
Aug 09 06:27:24 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.131.20@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
Aug 09 06:27:27 dist-test-slave-mjjr krb5kdc[25309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:27:27.291903 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:36427
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:36427
--ranger_config_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43459/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:27:27.646411 25970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:27:27.646782 25970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:27:27.646970 25970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:27:27.654475 25970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:27:27.654623 25970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:27:27.654728 25970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:27:27.654824 25970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:27:27.654914 25970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:27:27.665481 25970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.131.20:43459/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:36427
--ranger_config_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:36427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:27:27.666818 25970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:27:27.668253 25970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:27:27.681224 25976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:27:27.681334 25980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:27:27.681246 25978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:27:27.682057 25970 server_base.cc:1034] running on GCE node
I20260809 06:27:27.682930 25970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:27:27.684487 25970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:27:27.685686 25970 hybrid_clock.cc:648] HybridClock initialized: now 1786256847685632 us; error 53 us; skew 500 ppm
I20260809 06:27:27.686051 25970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:27:27 dist-test-slave-mjjr krb5kdc[25309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256847, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:27:27.689272 25970 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:27:27.690721 25970 webserver.cc:460] Webserver started at http://127.8.131.62:34149/ using document root <none> and password file <none>
I20260809 06:27:27.691363 25970 fs_manager.cc:359] Metadata directory not provided
I20260809 06:27:27.691529 25970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:27:27.691887 25970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 06:27:27 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256847, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:27:27.742722 25970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "058d0f6942f449c688411cdf16186481"
format_stamp: "Formatted at 2026-08-09 06:27:27 on dist-test-slave-mjjr"
server_key: "c4ff8889a5c711efc43a60a20ab549ee"
server_key_iv: "e8e6ab806e06c47f5d0d1fb84186ae9c"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:27.743870 25970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "058d0f6942f449c688411cdf16186481"
format_stamp: "Formatted at 2026-08-09 06:27:27 on dist-test-slave-mjjr"
server_key: "c4ff8889a5c711efc43a60a20ab549ee"
server_key_iv: "e8e6ab806e06c47f5d0d1fb84186ae9c"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:27.749765 25970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:27:27.766520 25988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:27:27.768198 25970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 06:27:27.768440 25970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "058d0f6942f449c688411cdf16186481"
format_stamp: "Formatted at 2026-08-09 06:27:27 on dist-test-slave-mjjr"
server_key: "c4ff8889a5c711efc43a60a20ab549ee"
server_key_iv: "e8e6ab806e06c47f5d0d1fb84186ae9c"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:27.768708 25970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:27:27.796593 25970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:27:27.801127 25970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:27:27.801506 25970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:27:27.821973 25970 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:36427
I20260809 06:27:27.822016 26056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:36427 every 8 connection(s)
I20260809 06:27:27.823340 25970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:27:27.826098 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 25970
I20260809 06:27:27.826327 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:27:27.827031 8716 external_mini_cluster.cc:1428] Setting key eed5a2a38fed3bc5ee104a88209f63c4
I20260809 06:27:27.827162 26057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 06:27:27 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:27:27.841588 26057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481: Bootstrap starting.
I20260809 06:27:27.844801 26057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481: Neither blocks nor log segments found. Creating new log.
I20260809 06:27:27.845901 26057 log.cc:824] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481: Log is configured to *not* fsync() on all Append() calls
I20260809 06:27:27.848716 26057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481: No bootstrap required, opened a new log
I20260809 06:27:27.849993 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:27.829140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43056 (local address 127.8.131.62:36427)
0809 06:27:27.830045 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:27.830056 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:27.830094 (+ 38us) server_negotiation.cc:407] Connection header received
0809 06:27:27.831821 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:27.831841 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:27.832220 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:27.832568 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:27:27.834126 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:27.835501 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:27.837326 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:27.838026 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:27.838157 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:27.842643 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:27:27.842650 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:27:27.842663 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:27:27.842697 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:27:27.845949 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:27.846654 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:27.846661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:27.846666 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:27.846825 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:27.847249 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:27.847254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:27.847257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:27.847832 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:27:27.848170 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:27.848611 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 06:27:27.849023 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":431,"threads_started":1}
I20260809 06:27:27.856225 26057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058d0f6942f449c688411cdf16186481" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36427 } }
I20260809 06:27:27.856598 26057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:27:27.856705 26057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058d0f6942f449c688411cdf16186481, State: Initialized, Role: FOLLOWER
I20260809 06:27:27.857339 26057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058d0f6942f449c688411cdf16186481" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36427 } }
I20260809 06:27:27.857656 26057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:27:27.857812 26057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:27:27.857998 26057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:27:27.860263 26057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058d0f6942f449c688411cdf16186481" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36427 } }
I20260809 06:27:27.860735 26057 leader_election.cc:302] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 058d0f6942f449c688411cdf16186481; no voters:
I20260809 06:27:27.861291 26057 leader_election.cc:288] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:27:27.861425 26063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:27:27.862543 26063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [term 1 LEADER]: Becoming Leader. State: Replica: 058d0f6942f449c688411cdf16186481, State: Running, Role: LEADER
I20260809 06:27:27.863078 26063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058d0f6942f449c688411cdf16186481" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36427 } }
I20260809 06:27:27.863844 26057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:27:27.871657 26064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "058d0f6942f449c688411cdf16186481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058d0f6942f449c688411cdf16186481" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36427 } } }
I20260809 06:27:27.871690 26066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 058d0f6942f449c688411cdf16186481. Latest consensus state: current_term: 1 leader_uuid: "058d0f6942f449c688411cdf16186481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058d0f6942f449c688411cdf16186481" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36427 } } }
I20260809 06:27:27.872169 26064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [sys.catalog]: This master's current role is: LEADER
I20260809 06:27:27.872192 26066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481 [sys.catalog]: This master's current role is: LEADER
I20260809 06:27:27.873906 26071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:27:27.891623 26071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:27:27.892659 26057 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 06:27:27.911406 26071 catalog_manager.cc:1347] Generated new cluster ID: 5eefa0a9d44a4e4b9b8f4348fa2f9529
I20260809 06:27:27.911614 26071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:27:27.932292 26071 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:27:27.934474 26071 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:27:27.956076 26071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481: Generated new TSK 0
I20260809 06:27:27.957245 26071 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:27:28.844115 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43459/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36427
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:27:29 dist-test-slave-mjjr krb5kdc[25309](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786256849, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 06:27:29.515304 26115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:27:29.515965 26115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:27:29.516275 26115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:27:29.527047 26115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:27:29.527309 26115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:27:29.527606 26115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:27:29.543174 26115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.131.20:43459/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:27:29.545090 26115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:27:29.547084 26115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:27:29.584198 26131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:27:29.586004 26130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:27:29.587828 26134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:27:29.589859 26115 server_base.cc:1034] running on GCE node
I20260809 06:27:29.590931 26115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:27:29.592653 26115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:27:29.594401 26115 hybrid_clock.cc:648] HybridClock initialized: now 1786256849594249 us; error 90 us; skew 500 ppm
I20260809 06:27:29.595124 26115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:27:29 dist-test-slave-mjjr krb5kdc[25309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256849, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:27:29.604579 26115 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:27:29.607817 26115 webserver.cc:460] Webserver started at http://127.8.131.1:44675/ using document root <none> and password file <none>
I20260809 06:27:29.608688 26115 fs_manager.cc:359] Metadata directory not provided
I20260809 06:27:29.608906 26115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:27:29.609464 26115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 06:27:29 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256849, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:27:29.639101 26115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "f46153ac5e8445708e0581a042b26f1d"
format_stamp: "Formatted at 2026-08-09 06:27:29 on dist-test-slave-mjjr"
server_key: "ea412c646342ea477cce4b61e1147a6c"
server_key_iv: "00ea58b9c3b9a78c21a7fc877760beea"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:29.640785 26115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "f46153ac5e8445708e0581a042b26f1d"
format_stamp: "Formatted at 2026-08-09 06:27:29 on dist-test-slave-mjjr"
server_key: "ea412c646342ea477cce4b61e1147a6c"
server_key_iv: "00ea58b9c3b9a78c21a7fc877760beea"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:29.658560 26115 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.005s sys 0.009s
I20260809 06:27:29.700819 26145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:27:29.703019 26115 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.006s sys 0.010s
I20260809 06:27:29.703303 26115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "f46153ac5e8445708e0581a042b26f1d"
format_stamp: "Formatted at 2026-08-09 06:27:29 on dist-test-slave-mjjr"
server_key: "ea412c646342ea477cce4b61e1147a6c"
server_key_iv: "00ea58b9c3b9a78c21a7fc877760beea"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:29.703730 26115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 06:27:29 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256849, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:27:29.769320 26115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:27:29.781405 26115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:27:29.782069 26115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:27:29.811272 26115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:27:29.837364 26115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:27:29.837677 26115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.001s
I20260809 06:27:29.838068 26115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:27:29.838271 26115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 06:27:30 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256849, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:27:30.377415 26115 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:35617
I20260809 06:27:30.377417 26301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:35617 every 8 connection(s)
I20260809 06:27:30.379245 26115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:27:30.385335 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 26115
I20260809 06:27:30.385573 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:27:30.386287 8716 external_mini_cluster.cc:1428] Setting key c06b064e4968c06d56e4614bcb3e5046
Aug 09 06:27:30 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256849, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:27:30.416921 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:30.383477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:48163 (local address 127.8.131.62:36427)
0809 06:27:30.384001 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:30.384007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:30.384068 (+ 61us) server_negotiation.cc:407] Connection header received
0809 06:27:30.386787 (+ 2719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:30.386794 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:30.386892 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:30.387025 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:27:30.391572 (+ 4547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:30.392787 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:30.396046 (+ 3259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:30.397656 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:30.397761 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:30.404357 (+ 6596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:27:30.404366 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:27:30.404371 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:27:30.404413 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:27:30.409677 (+ 5264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:30.411502 (+ 1825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:30.411508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:30.411512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:30.411795 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:30.413154 (+ 1359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:30.413160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:30.413164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:30.413691 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:27:30.413846 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:30.415515 (+ 1669us) server_negotiation.cc:299] Negotiation successful
0809 06:27:30.416287 (+ 772us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":913,"threads_started":1}
I20260809 06:27:30.418051 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:30.383043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:36427 (local address 127.8.131.1:48163)
0809 06:27:30.383924 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0809 06:27:30.383974 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 06:27:30.386411 (+ 2437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:27:30.387462 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:27:30.387476 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:27:30.390407 (+ 2931us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:27:30.391344 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:30.391374 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:30.392996 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:30.393002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:30.395851 (+ 2849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:30.395892 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:30.398263 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:30.398272 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:30.398655 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:30.399745 (+ 1090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:27:30.399778 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:27:30.403996 (+ 4218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:27:30.409870 (+ 5874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:27:30.409880 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:27:30.409899 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:27:30.411339 (+ 1440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:27:30.412551 (+ 1212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:27:30.412556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:27:30.412560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:27:30.412993 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:27:30.413992 (+ 999us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:27:30.414001 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:27:30.415059 (+ 1058us) client_negotiation.cc:769] Sending connection context
0809 06:27:30.415412 (+ 353us) client_negotiation.cc:240] Negotiation successful
0809 06:27:30.416101 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":555,"threads_started":1}
I20260809 06:27:30.420794 26304 heartbeater.cc:347] Connected to a master server at 127.8.131.62:36427
I20260809 06:27:30.421247 26304 heartbeater.cc:464] Registering TS with master...
I20260809 06:27:30.422372 26304 heartbeater.cc:511] Master 127.8.131.62:36427 requested a full tablet report, sending...
I20260809 06:27:30.425186 26010 ts_manager.cc:194] Registered new tserver with Master: f46153ac5e8445708e0581a042b26f1d (127.8.131.1:35617)
I20260809 06:27:30.427263 26010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:48163
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:27:30.478916 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43459/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36427
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:27:31.086529 26312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:27:31.086869 26312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:27:31.087102 26312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:27:31.096516 26312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:27:31.096710 26312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:27:31.096953 26312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:27:31.108896 26312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.131.20:43459/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:27:31.110772 26312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:27:31.112560 26312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:27:31.130048 26329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:27:31.130237 26330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:27:31.130313 26332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:27:31.131531 26312 server_base.cc:1034] running on GCE node
I20260809 06:27:31.132226 26312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:27:31.133415 26312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:27:31.134701 26312 hybrid_clock.cc:648] HybridClock initialized: now 1786256851134591 us; error 53 us; skew 500 ppm
I20260809 06:27:31.135110 26312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:27:31 dist-test-slave-mjjr krb5kdc[25309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256851, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:27:31.139394 26312 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:27:31.141463 26312 webserver.cc:460] Webserver started at http://127.8.131.2:35785/ using document root <none> and password file <none>
I20260809 06:27:31.142231 26312 fs_manager.cc:359] Metadata directory not provided
I20260809 06:27:31.142408 26312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:27:31.142869 26312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 06:27:31 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256851, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:27:31.158871 26312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d"
format_stamp: "Formatted at 2026-08-09 06:27:31 on dist-test-slave-mjjr"
server_key: "c7745785bd071574950561a9f6064d12"
server_key_iv: "4939934b8b59451839cc8fac60081ee5"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:31.159984 26312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d"
format_stamp: "Formatted at 2026-08-09 06:27:31 on dist-test-slave-mjjr"
server_key: "c7745785bd071574950561a9f6064d12"
server_key_iv: "4939934b8b59451839cc8fac60081ee5"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:31.166909 26312 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 06:27:31.182916 26346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:27:31.184563 26312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 06:27:31.184825 26312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d"
format_stamp: "Formatted at 2026-08-09 06:27:31 on dist-test-slave-mjjr"
server_key: "c7745785bd071574950561a9f6064d12"
server_key_iv: "4939934b8b59451839cc8fac60081ee5"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:31.185127 26312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:27:31.217044 26312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:27:31.222025 26312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:27:31.222497 26312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:27:31.223721 26312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:27:31.225802 26312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:27:31.225976 26312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:27:31.226190 26312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:27:31.226331 26312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:27:31.299990 26312 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:45395
I20260809 06:27:31.300071 26514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:45395 every 8 connection(s)
I20260809 06:27:31.301569 26312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:27:31.309854 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 26312
I20260809 06:27:31.310093 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:27:31.310633 8716 external_mini_cluster.cc:1428] Setting key ed5e7daf972d3f5ebf2f4b83dc2c6738
Aug 09 06:27:31 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256851, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:27:31.330745 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:31.304855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:36273 (local address 127.8.131.62:36427)
0809 06:27:31.305536 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:31.305540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:31.307120 (+ 1580us) server_negotiation.cc:407] Connection header received
0809 06:27:31.309471 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:31.309477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:31.309575 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:31.309708 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:27:31.311963 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:31.313034 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:31.316041 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:31.316621 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:31.316744 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:31.321963 (+ 5219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:27:31.321972 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:27:31.321979 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:27:31.322031 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:27:31.326582 (+ 4551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:31.327904 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:31.327908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:31.327910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:31.328096 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:31.328677 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:31.328681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:31.328683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:31.329032 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:27:31.329141 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:31.330168 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0809 06:27:31.330355 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":434,"threads_started":1}
I20260809 06:27:31.332317 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:31.305751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:36427 (local address 127.8.131.2:36273)
0809 06:27:31.306953 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0809 06:27:31.307005 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 06:27:31.309187 (+ 2182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:27:31.309966 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:27:31.309976 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:27:31.310696 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:27:31.311763 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:31.311787 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:31.313189 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:31.313195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:31.315868 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:31.315895 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:31.316919 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:31.316930 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:31.317363 (+ 433us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:31.318301 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:27:31.318335 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:27:31.321702 (+ 3367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:27:31.326767 (+ 5065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:27:31.326780 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:27:31.326800 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:27:31.327657 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:27:31.328212 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:27:31.328220 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:27:31.328226 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:27:31.328558 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:27:31.329162 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:27:31.329172 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:27:31.329645 (+ 473us) client_negotiation.cc:769] Sending connection context
0809 06:27:31.329976 (+ 331us) client_negotiation.cc:240] Negotiation successful
0809 06:27:31.330650 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"thread_start_us":631,"threads_started":1}
I20260809 06:27:31.333931 26515 heartbeater.cc:347] Connected to a master server at 127.8.131.62:36427
I20260809 06:27:31.334297 26515 heartbeater.cc:464] Registering TS with master...
I20260809 06:27:31.335230 26515 heartbeater.cc:511] Master 127.8.131.62:36427 requested a full tablet report, sending...
I20260809 06:27:31.337018 26010 ts_manager.cc:194] Registered new tserver with Master: 0f5a75a32e3a4b2293edcc8968dc2c3d (127.8.131.2:45395)
I20260809 06:27:31.337999 26010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:36273
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:27:31.378255 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43459/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36427
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:27:31.431259 26304 heartbeater.cc:503] Master 127.8.131.62:36427 was elected leader, sending a full tablet report...
W20260809 06:27:31.919574 26525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:27:31.920011 26525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:27:31.920217 26525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:27:31.929410 26525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:27:31.929611 26525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:27:31.929821 26525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:27:31.941799 26525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.131.20:43459/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:27:31.943271 26525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:27:31.944808 26525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:27:31.963858 26531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:27:31.963922 26533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:27:31.964098 26537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:27:31.965754 26525 server_base.cc:1034] running on GCE node
I20260809 06:27:31.966662 26525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:27:31.968000 26525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:27:31.969281 26525 hybrid_clock.cc:648] HybridClock initialized: now 1786256851969193 us; error 63 us; skew 500 ppm
I20260809 06:27:31.969681 26525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:27:31 dist-test-slave-mjjr krb5kdc[25309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256851, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:27:31.974407 26525 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:27:31.976934 26525 webserver.cc:460] Webserver started at http://127.8.131.3:44497/ using document root <none> and password file <none>
I20260809 06:27:31.977797 26525 fs_manager.cc:359] Metadata directory not provided
I20260809 06:27:31.978008 26525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:27:31.978467 26525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 06:27:31 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256851, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:27:31.996793 26525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "2af9c9e6b5bc4a4bbefb1c226333f577"
format_stamp: "Formatted at 2026-08-09 06:27:31 on dist-test-slave-mjjr"
server_key: "e74d55047f85fca85be9fa60b3c61955"
server_key_iv: "a178d6701ab2b8372fedac255dd7b0ee"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:31.998129 26525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "2af9c9e6b5bc4a4bbefb1c226333f577"
format_stamp: "Formatted at 2026-08-09 06:27:31 on dist-test-slave-mjjr"
server_key: "e74d55047f85fca85be9fa60b3c61955"
server_key_iv: "a178d6701ab2b8372fedac255dd7b0ee"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:32.006372 26525 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.005s
I20260809 06:27:32.023525 26549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:27:32.025134 26525 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 06:27:32.025398 26525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "2af9c9e6b5bc4a4bbefb1c226333f577"
format_stamp: "Formatted at 2026-08-09 06:27:31 on dist-test-slave-mjjr"
server_key: "e74d55047f85fca85be9fa60b3c61955"
server_key_iv: "a178d6701ab2b8372fedac255dd7b0ee"
server_key_version: "kuduclusterkey@0"
I20260809 06:27:32.025663 26525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:27:32.047348 26525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:27:32.052073 26525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:27:32.052482 26525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:27:32.053834 26525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:27:32.055819 26525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:27:32.055950 26525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:27:32.056100 26525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:27:32.056181 26525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:27:32.129954 26525 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:43381
I20260809 06:27:32.130110 26710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:43381 every 8 connection(s)
I20260809 06:27:32.131583 26525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:27:32.139106 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 26525
I20260809 06:27:32.139307 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:27:32.140080 8716 external_mini_cluster.cc:1428] Setting key cd677f2e55afd68271c3d04a99ec337f
Aug 09 06:27:32 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256851, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:27:32.157105 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.134814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:50803 (local address 127.8.131.62:36427)
0809 06:27:32.135743 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.135751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.136775 (+ 1024us) server_negotiation.cc:407] Connection header received
0809 06:27:32.139388 (+ 2613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.139392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.139504 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.139605 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:27:32.142114 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.143218 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.145365 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.145895 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.146034 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.150220 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:27:32.150224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:27:32.150228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:27:32.150260 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:27:32.153400 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.154217 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.154222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.154224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.154350 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.154874 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.154878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.154886 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.155270 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:27:32.155432 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.156304 (+ 872us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.156575 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":479,"threads_started":1}
I20260809 06:27:32.158195 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.135391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:36427 (local address 127.8.131.3:50803)
0809 06:27:32.136607 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0809 06:27:32.136653 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:27:32.138972 (+ 2319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:27:32.139969 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:27:32.139980 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:27:32.140857 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:27:32.141911 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.141937 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.143416 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.143420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.145185 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.145198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.146231 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.146250 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.146613 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.147337 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:27:32.147364 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:27:32.149975 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:27:32.153575 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:27:32.153584 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:27:32.153602 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:27:32.154082 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:27:32.154466 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:27:32.154472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:27:32.154475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:27:32.154742 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:27:32.155413 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:27:32.155421 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:27:32.155838 (+ 417us) client_negotiation.cc:769] Sending connection context
0809 06:27:32.156223 (+ 385us) client_negotiation.cc:240] Negotiation successful
0809 06:27:32.156856 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"thread_start_us":678,"threads_started":1}
I20260809 06:27:32.159725 26711 heartbeater.cc:347] Connected to a master server at 127.8.131.62:36427
I20260809 06:27:32.160113 26711 heartbeater.cc:464] Registering TS with master...
I20260809 06:27:32.161058 26711 heartbeater.cc:511] Master 127.8.131.62:36427 requested a full tablet report, sending...
I20260809 06:27:32.162669 26010 ts_manager.cc:194] Registered new tserver with Master: 2af9c9e6b5bc4a4bbefb1c226333f577 (127.8.131.3:43381)
I20260809 06:27:32.163594 26010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:50803
I20260809 06:27:32.168897 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:27:32.191322 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.175198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43072 (local address 127.8.131.62:36427)
0809 06:27:32.175367 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.175372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.176155 (+ 783us) server_negotiation.cc:407] Connection header received
0809 06:27:32.176310 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.176314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.176402 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.176485 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:27:32.178514 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.179444 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.181152 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.181865 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.182028 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.184806 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:27:32.184810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:27:32.184814 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:27:32.184857 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:27:32.188299 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.189029 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.189033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.189035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.189243 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.189738 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.189741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.189743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.190084 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:27:32.190309 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.190700 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.190922 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:27:32.196030 26010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:27:32.205538 26010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:27:32.247452 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.222421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.8.131.1:35617)
0809 06:27:32.235711 (+ 13290us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.235724 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.235762 (+ 38us) server_negotiation.cc:407] Connection header received
0809 06:27:32.235883 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.235894 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.236263 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.236528 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:27:32.238530 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.240684 (+ 2154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.244265 (+ 3581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.245556 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.245772 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.246138 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.246552 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.246768 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12982,"thread_start_us":12738,"threads_started":1}
I20260809 06:27:32.247570 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.221860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35617 (local address 127.0.0.1:54708)
0809 06:27:32.234252 (+ 12392us) negotiation.cc:107] Waiting for socket to connect
0809 06:27:32.234275 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:27:32.234452 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:27:32.236884 (+ 2432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:27:32.236896 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:27:32.236937 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:27:32.238234 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.238257 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.240872 (+ 2615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.240877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.244064 (+ 3187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.244080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.245905 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.245922 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.246216 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.246267 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 06:27:32.246511 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 06:27:32.246715 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12198,"thread_start_us":8827,"threads_started":1}
I20260809 06:27:32.250945 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.221071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:43381 (local address 127.0.0.1:52704)
0809 06:27:32.231255 (+ 10184us) negotiation.cc:107] Waiting for socket to connect
0809 06:27:32.231292 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:27:32.231542 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:27:32.238243 (+ 6701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:27:32.238246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:27:32.238272 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:27:32.240935 (+ 2663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.240946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.243733 (+ 2787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.243738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.246675 (+ 2937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.246688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.249782 (+ 3094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.249786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.249944 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.249967 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 06:27:32.250070 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 06:27:32.250199 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9864,"mutex_wait_us":3,"thread_start_us":19314,"threads_started":1}
I20260809 06:27:32.251051 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.221647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.8.131.3:43381)
0809 06:27:32.236505 (+ 14858us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.236518 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.236585 (+ 67us) server_negotiation.cc:407] Connection header received
0809 06:27:32.237756 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.237765 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.237983 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.238217 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:27:32.241292 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.243481 (+ 2189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.246860 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.249131 (+ 2271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.249235 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.249529 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.250147 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.250348 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14604,"thread_start_us":14342,"threads_started":1}
I20260809 06:27:32.253559 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.231721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:45395 (local address 127.0.0.1:45816)
0809 06:27:32.240879 (+ 9158us) negotiation.cc:107] Waiting for socket to connect
0809 06:27:32.240932 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 06:27:32.241112 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:27:32.241889 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:27:32.241909 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:27:32.241953 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:27:32.242925 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.242940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.245559 (+ 2619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.245565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.248644 (+ 3079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.248656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.252493 (+ 3837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.252500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.252714 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.252755 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 06:27:32.252855 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 06:27:32.252988 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8997,"thread_start_us":2874,"threads_started":1}
I20260809 06:27:32.253746 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.232238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.8.131.2:45395)
0809 06:27:32.237453 (+ 5215us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.237462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.241074 (+ 3612us) server_negotiation.cc:407] Connection header received
0809 06:27:32.241399 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.241407 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.241654 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.241862 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:27:32.243158 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.245377 (+ 2219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.248827 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.251828 (+ 3001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.252005 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.252221 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.252949 (+ 728us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.253156 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4975,"thread_start_us":4765,"threads_started":1}
I20260809 06:27:32.253278 26212 tablet_service.cc:1467] Processing CreateTablet for tablet 166251a1f6374bc7bfce748fd16bd04c (DEFAULT_TABLE table=test-table [id=0c552a3d175e415c8d19642765e2c21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:27:32.255384 26212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166251a1f6374bc7bfce748fd16bd04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:27:32.255688 26617 tablet_service.cc:1467] Processing CreateTablet for tablet 166251a1f6374bc7bfce748fd16bd04c (DEFAULT_TABLE table=test-table [id=0c552a3d175e415c8d19642765e2c21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:27:32.257855 26617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166251a1f6374bc7bfce748fd16bd04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:27:32.258214 26413 tablet_service.cc:1467] Processing CreateTablet for tablet 166251a1f6374bc7bfce748fd16bd04c (DEFAULT_TABLE table=test-table [id=0c552a3d175e415c8d19642765e2c21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:27:32.259815 26413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166251a1f6374bc7bfce748fd16bd04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:27:32.269753 26737 tablet_bootstrap.cc:492] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: Bootstrap starting.
I20260809 06:27:32.275863 26737 tablet_bootstrap.cc:654] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: Neither blocks nor log segments found. Creating new log.
I20260809 06:27:32.277657 26737 log.cc:824] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: Log is configured to *not* fsync() on all Append() calls
I20260809 06:27:32.281154 26742 tablet_bootstrap.cc:492] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: Bootstrap starting.
I20260809 06:27:32.282505 26737 tablet_bootstrap.cc:492] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: No bootstrap required, opened a new log
I20260809 06:27:32.282975 26737 ts_tablet_manager.cc:1397] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260809 06:27:32.284261 26740 tablet_bootstrap.cc:492] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: Bootstrap starting.
I20260809 06:27:32.287068 26742 tablet_bootstrap.cc:654] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: Neither blocks nor log segments found. Creating new log.
I20260809 06:27:32.289059 26740 tablet_bootstrap.cc:654] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: Neither blocks nor log segments found. Creating new log.
I20260809 06:27:32.289319 26742 log.cc:824] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: Log is configured to *not* fsync() on all Append() calls
I20260809 06:27:32.290871 26740 log.cc:824] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: Log is configured to *not* fsync() on all Append() calls
I20260809 06:27:32.294541 26740 tablet_bootstrap.cc:492] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: No bootstrap required, opened a new log
I20260809 06:27:32.294852 26740 ts_tablet_manager.cc:1397] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.008s
I20260809 06:27:32.299829 26742 tablet_bootstrap.cc:492] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: No bootstrap required, opened a new log
I20260809 06:27:32.300228 26742 ts_tablet_manager.cc:1397] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.018s
I20260809 06:27:32.299883 26737 raft_consensus.cc:348] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } }
I20260809 06:27:32.300457 26737 raft_consensus.cc:374] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:27:32.300653 26737 raft_consensus.cc:729] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46153ac5e8445708e0581a042b26f1d, State: Initialized, Role: FOLLOWER
I20260809 06:27:32.301375 26737 consensus_queue.cc:260] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } }
I20260809 06:27:32.309974 26740 raft_consensus.cc:348] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } }
I20260809 06:27:32.310684 26740 raft_consensus.cc:374] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:27:32.310886 26740 raft_consensus.cc:729] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2af9c9e6b5bc4a4bbefb1c226333f577, State: Initialized, Role: FOLLOWER
I20260809 06:27:32.311856 26740 consensus_queue.cc:260] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } }
I20260809 06:27:32.320420 26711 heartbeater.cc:503] Master 127.8.131.62:36427 was elected leader, sending a full tablet report...
I20260809 06:27:32.323305 26737 ts_tablet_manager.cc:1428] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: Time spent starting tablet: real 0.040s user 0.019s sys 0.017s
I20260809 06:27:32.323890 26750 raft_consensus.cc:482] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:27:32.324292 26750 raft_consensus.cc:504] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } }
I20260809 06:27:32.325112 26740 ts_tablet_manager.cc:1428] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: Time spent starting tablet: real 0.030s user 0.008s sys 0.017s
I20260809 06:27:32.325472 26742 raft_consensus.cc:348] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } }
I20260809 06:27:32.326447 26742 raft_consensus.cc:374] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:27:32.326694 26742 raft_consensus.cc:729] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5a75a32e3a4b2293edcc8968dc2c3d, State: Initialized, Role: FOLLOWER
I20260809 06:27:32.327781 26742 consensus_queue.cc:260] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } }
I20260809 06:27:32.335160 26515 heartbeater.cc:503] Master 127.8.131.62:36427 was elected leader, sending a full tablet report...
I20260809 06:27:32.336359 26742 ts_tablet_manager.cc:1428] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: Time spent starting tablet: real 0.036s user 0.011s sys 0.018s
I20260809 06:27:32.340716 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.327634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35617 (local address 127.8.131.3:48705)
0809 06:27:32.329791 (+ 2157us) negotiation.cc:107] Waiting for socket to connect
0809 06:27:32.329811 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:27:32.329922 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:27:32.331006 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:27:32.331024 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:27:32.331052 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:27:32.331430 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.331447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.333185 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.333191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.336254 (+ 3063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.336272 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.339774 (+ 3502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.339780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.340055 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.340089 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:27:32.340204 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 06:27:32.340332 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2000,"mutex_wait_us":4}
I20260809 06:27:32.342584 26750 leader_election.cc:288] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46153ac5e8445708e0581a042b26f1d (127.8.131.1:35617), 0f5a75a32e3a4b2293edcc8968dc2c3d (127.8.131.2:45395)
I20260809 06:27:32.343807 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.330275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:48705 (local address 127.8.131.1:35617)
0809 06:27:32.330473 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.330478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.330497 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:27:32.330558 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.330578 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.330660 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.330828 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:27:32.331627 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.332989 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.336520 (+ 3531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.338370 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.338569 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.338748 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.343025 (+ 4277us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.343406 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 06:27:32.345120 26245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166251a1f6374bc7bfce748fd16bd04c" candidate_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46153ac5e8445708e0581a042b26f1d" is_pre_election: true
I20260809 06:27:32.346006 26245 raft_consensus.cc:2454] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2af9c9e6b5bc4a4bbefb1c226333f577 in term 0.
I20260809 06:27:32.346979 26554 leader_election.cc:302] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2af9c9e6b5bc4a4bbefb1c226333f577, f46153ac5e8445708e0581a042b26f1d; no voters:
I20260809 06:27:32.347518 26750 raft_consensus.cc:2781] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:27:32.347744 26750 raft_consensus.cc:482] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:27:32.347890 26750 raft_consensus.cc:3037] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:27:32.349310 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.338509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:49647 (local address 127.8.131.2:45395)
0809 06:27:32.338745 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.338751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.341017 (+ 2266us) server_negotiation.cc:407] Connection header received
0809 06:27:32.341198 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.341202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.341280 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.341362 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:27:32.342419 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.343499 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.346501 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.348235 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.348357 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.348422 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.349000 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.349086 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:27:32.349583 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.338119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:45395 (local address 127.8.131.3:49647)
0809 06:27:32.340937 (+ 2818us) negotiation.cc:107] Waiting for socket to connect
0809 06:27:32.340955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:27:32.341063 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:27:32.341610 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:27:32.341614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:27:32.341667 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:27:32.342157 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.342190 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.343733 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.343738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.346240 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:27:32.346257 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.348462 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.348468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:27:32.348812 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.348846 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:27:32.348949 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 06:27:32.349134 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2735,"thread_start_us":3955,"threads_started":1}
I20260809 06:27:32.350951 26443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166251a1f6374bc7bfce748fd16bd04c" candidate_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" is_pre_election: true
I20260809 06:27:32.351634 26443 raft_consensus.cc:2454] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2af9c9e6b5bc4a4bbefb1c226333f577 in term 0.
I20260809 06:27:32.351842 26750 raft_consensus.cc:504] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } }
I20260809 06:27:32.352730 26750 leader_election.cc:288] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [CANDIDATE]: Term 1 election: Requested vote from peers f46153ac5e8445708e0581a042b26f1d (127.8.131.1:35617), 0f5a75a32e3a4b2293edcc8968dc2c3d (127.8.131.2:45395)
I20260809 06:27:32.353297 26443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166251a1f6374bc7bfce748fd16bd04c" candidate_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d"
I20260809 06:27:32.353310 26245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166251a1f6374bc7bfce748fd16bd04c" candidate_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46153ac5e8445708e0581a042b26f1d"
I20260809 06:27:32.353531 26443 raft_consensus.cc:3037] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:27:32.353547 26245 raft_consensus.cc:3037] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:27:32.356490 26443 raft_consensus.cc:2454] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2af9c9e6b5bc4a4bbefb1c226333f577 in term 1.
I20260809 06:27:32.356667 26245 raft_consensus.cc:2454] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2af9c9e6b5bc4a4bbefb1c226333f577 in term 1.
I20260809 06:27:32.356923 26554 leader_election.cc:302] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5a75a32e3a4b2293edcc8968dc2c3d, 2af9c9e6b5bc4a4bbefb1c226333f577; no voters:
I20260809 06:27:32.357297 26750 raft_consensus.cc:2781] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:27:32.357553 26750 raft_consensus.cc:686] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 1 LEADER]: Becoming Leader. State: Replica: 2af9c9e6b5bc4a4bbefb1c226333f577, State: Running, Role: LEADER
I20260809 06:27:32.358222 26750 consensus_queue.cc:237] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } }
I20260809 06:27:32.363229 26009 catalog_manager.cc:5495] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2af9c9e6b5bc4a4bbefb1c226333f577 (127.8.131.3). New cstate: current_term: 1 leader_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af9c9e6b5bc4a4bbefb1c226333f577" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 43381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 } health_report { overall_health: UNKNOWN } } }
W20260809 06:27:32.382645 26305 tablet.cc:2369] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:27:32.383975 26712 tablet.cc:2369] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:27:32.392068 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.384905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.8.131.3:43381)
0809 06:27:32.385164 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.385170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.385190 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:27:32.385348 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.385354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.385494 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.385663 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:27:32.386055 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.387483 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.389764 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.390314 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.390418 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.391079 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:27:32.391564 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:27:32.391754 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.391802 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.391861 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 06:27:32.401216 26245 raft_consensus.cc:1261] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [term 1 FOLLOWER]: Refusing update from remote peer 2af9c9e6b5bc4a4bbefb1c226333f577: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:27:32.401563 26443 raft_consensus.cc:1261] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [term 1 FOLLOWER]: Refusing update from remote peer 2af9c9e6b5bc4a4bbefb1c226333f577: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:27:32.402340 26751 consensus_queue.cc:1035] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46153ac5e8445708e0581a042b26f1d" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:27:32.402851 26750 consensus_queue.cc:1035] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f5a75a32e3a4b2293edcc8968dc2c3d" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:27:32.418268 26762 mvcc.cc:205] Tried to move back new op lower bound from 7316508067425386496 to 7316508067263332352. Current Snapshot: MvccSnapshot[applied={T|T < 7316508067425386496}]
I20260809 06:27:32.437114 26010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:27:32.437522 26010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:27:32.441918 26010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 058d0f6942f449c688411cdf16186481: Sending DeleteTablet for 3 replicas of tablet 166251a1f6374bc7bfce748fd16bd04c
I20260809 06:27:32.443466 26617 tablet_service.cc:1514] Processing DeleteTablet for tablet 166251a1f6374bc7bfce748fd16bd04c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:27:32 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:52704
I20260809 06:27:32.443538 26212 tablet_service.cc:1514] Processing DeleteTablet for tablet 166251a1f6374bc7bfce748fd16bd04c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:27:32 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:54708
I20260809 06:27:32.443877 26413 tablet_service.cc:1514] Processing DeleteTablet for tablet 166251a1f6374bc7bfce748fd16bd04c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:27:32 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:45816
I20260809 06:27:32.447041 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 26115
I20260809 06:27:32.455928 26775 tablet_replica.cc:330] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: stopping tablet replica
I20260809 06:27:32.456743 26775 raft_consensus.cc:2229] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:27:32.456836 26778 tablet_replica.cc:330] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: stopping tablet replica
I20260809 06:27:32.457393 26775 raft_consensus.cc:2258] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:27:32.457597 26778 raft_consensus.cc:2229] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:27:32.458840 26778 raft_consensus.cc:2258] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:27:32.459936 26781 tablet_replica.cc:330] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: stopping tablet replica
I20260809 06:27:32.460213 26775 ts_tablet_manager.cc:1905] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:27:32.462110 26778 ts_tablet_manager.cc:1905] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:27:32.463246 26781 raft_consensus.cc:2229] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:27:32.465137 26781 raft_consensus.cc:2258] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:27:32.467799 26775 ts_tablet_manager.cc:1918] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:27:32.468048 26775 log.cc:1196] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/166251a1f6374bc7bfce748fd16bd04c
I20260809 06:27:32.468715 26775 ts_tablet_manager.cc:1939] T 166251a1f6374bc7bfce748fd16bd04c P f46153ac5e8445708e0581a042b26f1d: Deleting consensus metadata
I20260809 06:27:32.469094 26781 ts_tablet_manager.cc:1905] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:27:32.470268 26778 ts_tablet_manager.cc:1918] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:27:32.470491 26778 log.cc:1196] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/166251a1f6374bc7bfce748fd16bd04c
I20260809 06:27:32.470880 25993 catalog_manager.cc:4841] TS f46153ac5e8445708e0581a042b26f1d (127.8.131.1:35617): tablet 166251a1f6374bc7bfce748fd16bd04c (table test-table [id=0c552a3d175e415c8d19642765e2c21b]) successfully deleted
I20260809 06:27:32.471299 26778 ts_tablet_manager.cc:1939] T 166251a1f6374bc7bfce748fd16bd04c P 2af9c9e6b5bc4a4bbefb1c226333f577: Deleting consensus metadata
I20260809 06:27:32.473605 25994 catalog_manager.cc:4841] TS 2af9c9e6b5bc4a4bbefb1c226333f577 (127.8.131.3:43381): tablet 166251a1f6374bc7bfce748fd16bd04c (table test-table [id=0c552a3d175e415c8d19642765e2c21b]) successfully deleted
Aug 09 06:27:32 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:27:32.477420 26781 ts_tablet_manager.cc:1918] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:27:32.477736 26781 log.cc:1196] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/166251a1f6374bc7bfce748fd16bd04c
I20260809 06:27:32.478487 26781 ts_tablet_manager.cc:1939] T 166251a1f6374bc7bfce748fd16bd04c P 0f5a75a32e3a4b2293edcc8968dc2c3d: Deleting consensus metadata
I20260809 06:27:32.480894 25993 catalog_manager.cc:4841] TS 0f5a75a32e3a4b2293edcc8968dc2c3d (127.8.131.2:45395): tablet 166251a1f6374bc7bfce748fd16bd04c (table test-table [id=0c552a3d175e415c8d19642765e2c21b]) successfully deleted
I20260809 06:27:32.485944 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.449182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.8.131.1:35617)
0809 06:27:32.449419 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.449428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.462416 (+ 12988us) server_negotiation.cc:407] Connection header received
0809 06:27:32.462663 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.462668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.462768 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.462892 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:27:32.467784 (+ 4892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.468941 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.471065 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.471945 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.472048 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.477982 (+ 5934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:27:32.477996 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:27:32.478011 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:27:32.478046 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:27:32.482428 (+ 4382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.483347 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.483356 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.483360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.483531 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.484040 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.484045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.484048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.484518 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:27:32.484992 (+ 474us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.485254 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.485527 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 06:27:32.487112 26296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:27:32.582648 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 26115
I20260809 06:27:32.610813 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 26312
Aug 09 06:27:32 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:27:32.628579 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.611892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45826 (local address 127.8.131.2:45395)
0809 06:27:32.612073 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.612078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.612096 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:27:32.612151 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.612154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.612232 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.612317 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:27:32.613994 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.614715 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.616614 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.616988 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.617082 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.622439 (+ 5357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:27:32.622455 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:27:32.622471 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:27:32.622511 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:27:32.625552 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.626317 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.626325 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.626327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.626485 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.626901 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.626910 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.626912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.627287 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:27:32.627615 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.627994 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.628223 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 06:27:32.629264 26507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:27:32.720857 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 26312
I20260809 06:27:32.746361 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 26525
Aug 09 06:27:32 dist-test-slave-mjjr krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:27:32.764142 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:27:32.747194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52732 (local address 127.8.131.3:43381)
0809 06:27:32.747367 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 06:27:32.747375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:27:32.747435 (+ 60us) server_negotiation.cc:407] Connection header received
0809 06:27:32.747713 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:27:32.747718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:27:32.747790 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:27:32.747918 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:27:32.749259 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.750093 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.751452 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:27:32.752175 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:27:32.752336 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:27:32.756827 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:27:32.756842 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:27:32.756861 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:27:32.756909 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:27:32.761033 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.761774 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.761784 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.761788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.761978 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:27:32.762375 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:27:32.762381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:27:32.762384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:27:32.762864 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:27:32.763206 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 06:27:32.763382 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 06:27:32.763712 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:27:32.765103 26706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:27:32.900830 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 26525
I20260809 06:27:32.926100 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 25970
I20260809 06:27:32.927371 26051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:27:33.044288 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 25970
I20260809 06:27:33.075235 8716 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 06:27:33.421118 8716 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 06:27:33.421442 8716 mini_ranger.cc:66] Stopping Ranger...
I20260809 06:27:33.767028 8716 mini_ranger.cc:68] Stopped Ranger
2026-08-09 06:27:33.767 UTC [25356] LOG: received smart shutdown request
2026-08-09 06:27:33.771 UTC [25356] LOG: background worker "logical replication launcher" (PID 25367) exited with exit code 1
2026-08-09 06:27:33.771 UTC [25362] LOG: shutting down
2026-08-09 06:27:33.911 UTC [25356] LOG: database system is shut down
2026-08-09T06:27:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51681 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:27:34 dist-test-slave-mjjr krb5kdc[26801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:27:34 dist-test-slave-mjjr krb5kdc[26801](info): set up 2 sockets
Aug 09 06:27:34 dist-test-slave-mjjr krb5kdc[26801](info): commencing operation
krb5kdc: starting...
W20260809 06:27:35.361238 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:27:35 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:27:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:27:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.131.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.131.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/rangeradmin_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/rangeradmin_127.8.131.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.131.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.131.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/rangerlookup_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/rangerlookup_127.8.131.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.131.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.131.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/HTTP_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/HTTP_127.8.131.20@KRBTEST.COM.keytab.
I20260809 06:27:35.724692 8716 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskn2CWnW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/postgres -l logfile start
2026-08-09 06:27:38.062 UTC [26843] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 06:27:38.062 UTC [26843] LOG: listening on IPv4 address "127.8.131.20", port 49747
2026-08-09 06:27:38.066 UTC [26843] LOG: listening on Unix socket "/tmp/.s.PGSQL.49747"
2026-08-09 06:27:38.089 UTC [26846] LOG: database system was shut down at 2026-08-09 06:27:36 UTC
2026-08-09 06:27:38.093 UTC [26843] LOG: database system is ready to accept connections
I20260809 06:27:39.278630 8716 mini_postgres.cc:96] Postgres bound to 49747
2026-08-09 06:27:39.300 UTC [26855] FATAL: database "slave" does not exist
127.8.131.20:49747 - accepting connections
I20260809 06:27:39.301191 8716 mini_ranger.cc:161] Starting Ranger...
I20260809 06:27:39.328814 8716 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 06:27:39.771469 8716 mini_ranger.cc:87] Created ranger Postgres database
I20260809 06:27:39.771575 8716 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-admin
2026-08-09 06:27:40,065 [I] DB FLAVOR :POSTGRES
2026-08-09 06:27:40,066 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 06:27:40,066 [I] Checking connection..
2026-08-09 06:27:40,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 06:27:40,352 [I] Checking connection passed.
2026-08-09 06:27:40,352 [I] --------- Verifying version history table ---------
2026-08-09 06:27:40,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 06:27:40,622 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 06:27:40,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 06:27:40,893 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 06:27:40,894 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 06:27:40,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 06:27:41.156 UTC [26936] WARNING: there is no transaction in progress
2026-08-09 06:27:41,165 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 06:27:41,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 06:27:41,420 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 06:27:41,420 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 06:27:41,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 06:27:41,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 06:27:41,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mjjr.c.gcp-upstream.internal','N') ;"
2026-08-09 06:27:42,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 06:27:42,433 [I] Table x_portal_user does not exist in database ranger
2026-08-09 06:27:42,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 06:27:42,688 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 06:27:42,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 06:27:42,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 06:27:43,183 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 06:27:43,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 06:27:44.105 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.120 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.135 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.149 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.166 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.183 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.235 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.241 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.254 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.263 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.272 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.285 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.294 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.300 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.306 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44.312 UTC [27081] WARNING: there is no transaction in progress
2026-08-09 06:27:44,911 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 06:27:44,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 06:27:45,158 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 06:27:45,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 06:27:45,423 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 06:27:45,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 06:27:45,676 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 06:27:45,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 06:27:45,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 06:27:46,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mjjr.c.gcp-upstream.internal';"
2026-08-09 06:27:46,426 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 06:27:46,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 06:27:46,662 [I] DB_PATCHES have already been applied
I20260809 06:27:46.669132 8716 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-admin:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskn2CWnW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 06:27:46.669227 8716 mini_ranger.cc:193] Using host: 127.8.131.20
I20260809 06:27:46.688678 8716 mini_ranger.cc:227] Ranger admin URL: http://127.8.131.20:60033
Aug 09, 2026 6:27:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 6:27:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 6:27:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 6:27:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 6:27:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 6:27:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 6:27:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 06:27:48.102572 8716 mini_ranger.cc:160] Time spent starting Ranger: real 8.801s user 0.000s sys 0.053s
Aug 09, 2026 6:28:08 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 6:28:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 6:28:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 6:28:08 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 6:28:09 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 06:28:10.310915 8716 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.131.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.131.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/rangerkms_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.131.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/rangerkms_127.8.131.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.131.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.131.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.131.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/HTTP_127.8.131.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.131.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/HTTP_127.8.131.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 06:28:10 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256890, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 06:28:10.520457 8716 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 06:28:10.544324 8716 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 06:28:11.353230 8716 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 06:28:11.353365 8716 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-kms
I20260809 06:28:11.353410 8716 mini_ranger_kms.cc:227] Using postgres at 127.8.131.20:49747
2026-08-09 06:28:11,726 [I] DB FLAVOR :POSTGRES
2026-08-09 06:28:11,726 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 06:28:11,726 [I] Checking connection
2026-08-09 06:28:11,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 06:28:11,982 [I] connection success
2026-08-09 06:28:11,982 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 06:28:11,982 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 06:28:11,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 06:28:12,253 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 06:28:12,253 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 06:28:12,253 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 06:28:12,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.131.20:49747/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 06:28:12,518 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 06:28:12.991158 8716 mini_ranger_kms.cc:310] Created kms service
I20260809 06:28:13.235955 8716 mini_ranger_kms.cc:326] Created kudu user
I20260809 06:28:13.282256 8716 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 06:28:13 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256890, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:13.690196 8716 mini_ranger_kms.cc:384] Added ranger policy
I20260809 06:28:13.690517 8716 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-kms:/tmp/dist-test-taskn2CWnW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskn2CWnW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 06:28:13.690559 8716 mini_ranger_kms.cc:240] Using host: 127.8.131.20
I20260809 06:28:13.710968 8716 mini_ranger_kms.cc:276] Ranger KMS PID: 27338
I20260809 06:28:13.711110 8716 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.131.20:43125
Aug 09, 2026 6:28:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 6:28:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 6:28:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 6:28:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 6:28:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 6:28:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 06:28:15 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786256895, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.131.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 06:28:16 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256895, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.131.20@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:16.562916 8716 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.042s user 0.068s sys 0.331s
I20260809 06:28:16.563207 8716 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 06:28:16.563403 8716 mini_ranger_kms.cc:401] 127.8.131.20:43125/kms/v1/keys
Aug 09 06:28:16 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256895, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.131.20@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
Aug 09 06:28:18 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:28:18.917519 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41969
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:41969
--ranger_config_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43125/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:19.262279 27403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:19.262565 27403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:19.262655 27403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:19.269922 27403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:28:19.270035 27403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:19.270080 27403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:19.270129 27403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:28:19.270182 27403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:28:19.280014 27403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.131.20:43125/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:41969
--ranger_config_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:19.281195 27403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:19.282444 27403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:19.294380 27411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:19.294360 27412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:19.294662 27414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:19.295686 27403 server_base.cc:1034] running on GCE node
I20260809 06:28:19.296521 27403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:19.297887 27403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:19.299110 27403 hybrid_clock.cc:648] HybridClock initialized: now 1786256899299080 us; error 41 us; skew 500 ppm
I20260809 06:28:19.299456 27403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:19 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256899, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:19.302536 27403 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:28:19.304018 27403 webserver.cc:460] Webserver started at http://127.8.131.62:43837/ using document root <none> and password file <none>
I20260809 06:28:19.304654 27403 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:19.304803 27403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:19.305109 27403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 06:28:19 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256899, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:19.354441 27403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "af46194188084aef95fe79109e1ec8eb"
format_stamp: "Formatted at 2026-08-09 06:28:19 on dist-test-slave-mjjr"
server_key: "1be4b1312fb1d35f40bd188e3fe0e42d"
server_key_iv: "3102de1ec2b6f47a99a839c48373ebaa"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:19.355347 27403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "af46194188084aef95fe79109e1ec8eb"
format_stamp: "Formatted at 2026-08-09 06:28:19 on dist-test-slave-mjjr"
server_key: "1be4b1312fb1d35f40bd188e3fe0e42d"
server_key_iv: "3102de1ec2b6f47a99a839c48373ebaa"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:19.360797 27403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 06:28:19.378127 27425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:19.380038 27403 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 06:28:19.380256 27403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "af46194188084aef95fe79109e1ec8eb"
format_stamp: "Formatted at 2026-08-09 06:28:19 on dist-test-slave-mjjr"
server_key: "1be4b1312fb1d35f40bd188e3fe0e42d"
server_key_iv: "3102de1ec2b6f47a99a839c48373ebaa"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:19.380535 27403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:19.400326 27403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:19.405193 27403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:19.405570 27403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:19.425405 27403 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:41969
I20260809 06:28:19.425477 27502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:41969 every 8 connection(s)
I20260809 06:28:19.427205 27403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:28:19.431094 27503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:19.436051 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 27403
I20260809 06:28:19.436290 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:28:19.437103 8716 external_mini_cluster.cc:1428] Setting key 31ce9b1b059bf9756a9732a415cace07
I20260809 06:28:19.444471 27503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Bootstrap starting.
I20260809 06:28:19.449656 27503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:19.451444 27503 log.cc:824] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:19.455708 27503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: No bootstrap required, opened a new log
Aug 09 06:28:19 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:19.467101 27503 raft_consensus.cc:348] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } }
I20260809 06:28:19.467612 27503 raft_consensus.cc:374] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:19.467753 27503 raft_consensus.cc:729] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af46194188084aef95fe79109e1ec8eb, State: Initialized, Role: FOLLOWER
I20260809 06:28:19.468464 27503 consensus_queue.cc:260] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } }
I20260809 06:28:19.468737 27503 raft_consensus.cc:388] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:28:19.468878 27503 raft_consensus.cc:482] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:28:19.469035 27503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:19.472368 27503 raft_consensus.cc:504] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } }
I20260809 06:28:19.472802 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:19.439356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43392 (local address 127.8.131.62:41969)
0809 06:28:19.452261 (+ 12905us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:19.452279 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:19.452902 (+ 623us) server_negotiation.cc:407] Connection header received
0809 06:28:19.454927 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:19.454951 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:19.455306 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:19.455711 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:19.457166 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:19.458520 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:19.459963 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:19.460660 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:19.460851 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:19.464928 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:19.464937 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:19.464952 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:19.464992 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:19.468299 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:19.469115 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:19.469127 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:19.469138 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:19.469306 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:19.469765 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:19.469770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:19.469774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:19.470342 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:19.470684 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:19.471126 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 06:28:19.471722 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12636,"thread_start_us":12399,"threads_started":1}
I20260809 06:28:19.472959 27503 leader_election.cc:302] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af46194188084aef95fe79109e1ec8eb; no voters:
I20260809 06:28:19.473699 27503 leader_election.cc:288] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:28:19.473829 27511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:28:19.477484 27511 raft_consensus.cc:686] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 1 LEADER]: Becoming Leader. State: Replica: af46194188084aef95fe79109e1ec8eb, State: Running, Role: LEADER
I20260809 06:28:19.478103 27511 consensus_queue.cc:237] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } }
I20260809 06:28:19.479929 27511 sys_catalog.cc:455] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader af46194188084aef95fe79109e1ec8eb. Latest consensus state: current_term: 1 leader_uuid: "af46194188084aef95fe79109e1ec8eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } } }
I20260809 06:28:19.481972 27503 sys_catalog.cc:564] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:28:19.483402 27511 sys_catalog.cc:458] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:19.484318 27513 sys_catalog.cc:455] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af46194188084aef95fe79109e1ec8eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } } }
I20260809 06:28:19.484705 27513 sys_catalog.cc:458] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:19.491094 27518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:28:19.491382 27503 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 06:28:19.504117 27518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:28:19.515619 27518 catalog_manager.cc:1347] Generated new cluster ID: 650679e286ee4e2c9886561274344316
I20260809 06:28:19.515771 27518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:28:19.526057 27518 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:28:19.527915 27518 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:28:19.537470 27518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Generated new TSK 0
I20260809 06:28:19.538363 27518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:28:20.342988 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:41969
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:28:20 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786256900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 06:28:20.756659 27568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:20.757077 27568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:20.757253 27568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:20.767225 27568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:20.767423 27568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:20.767668 27568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:28:20.782346 27568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.131.20:43125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:20.784225 27568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:20.785621 27568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:20.803685 27581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:20.804792 27582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:20.813684 27586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:20.813688 27568 server_base.cc:1034] running on GCE node
I20260809 06:28:20.814667 27568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:20.815752 27568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:20.817028 27568 hybrid_clock.cc:648] HybridClock initialized: now 1786256900816957 us; error 72 us; skew 500 ppm
I20260809 06:28:20.817366 27568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:20 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:20.820711 27568 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:28:20.822594 27568 webserver.cc:460] Webserver started at http://127.8.131.1:46551/ using document root <none> and password file <none>
I20260809 06:28:20.823304 27568 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:20.823450 27568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:20.823850 27568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 06:28:20 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:20.843997 27568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "2d638e4c7e874680ba2594305c2175a9"
format_stamp: "Formatted at 2026-08-09 06:28:20 on dist-test-slave-mjjr"
server_key: "ea975348c704713d82b0f651375c42da"
server_key_iv: "c63f22c21db6e9e1d0cf477310bad1ec"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:20.845160 27568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "2d638e4c7e874680ba2594305c2175a9"
format_stamp: "Formatted at 2026-08-09 06:28:20 on dist-test-slave-mjjr"
server_key: "ea975348c704713d82b0f651375c42da"
server_key_iv: "c63f22c21db6e9e1d0cf477310bad1ec"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:20.855264 27568 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.003s sys 0.007s
I20260809 06:28:20.888319 27596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:20.890457 27568 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.007s sys 0.004s
I20260809 06:28:20.890736 27568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "2d638e4c7e874680ba2594305c2175a9"
format_stamp: "Formatted at 2026-08-09 06:28:20 on dist-test-slave-mjjr"
server_key: "ea975348c704713d82b0f651375c42da"
server_key_iv: "c63f22c21db6e9e1d0cf477310bad1ec"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:20.891121 27568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:20.953801 27568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:20.959491 27568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:20.960003 27568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:20.967689 27568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:20.978298 27568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:20.978520 27568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 06:28:20.978739 27568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:20.978902 27568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:21.154758 27757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:38741 every 8 connection(s)
I20260809 06:28:21.155194 27568 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:38741
I20260809 06:28:21.157066 27568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:28:21.162231 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 27568
I20260809 06:28:21.162458 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:28:21.163290 8716 external_mini_cluster.cc:1428] Setting key c0bd7962ed2e5b17a89adc7b1d7668f0
Aug 09 06:28:21 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
I20260809 06:28:21.231873 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:21.180616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:32883 (local address 127.8.131.62:41969)
0809 06:28:21.204875 (+ 24259us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:21.204884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:21.204914 (+ 30us) server_negotiation.cc:407] Connection header received
0809 06:28:21.204989 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:21.204998 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:21.205109 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:21.205264 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:21.207624 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:21.209010 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:21.211917 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:21.212736 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:21.212898 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:21.223458 (+ 10560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:21.223469 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:21.223478 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:21.223534 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:21.227895 (+ 4361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:21.228851 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:21.228859 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:21.228862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:21.229039 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:21.229571 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:21.229579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:21.229585 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:21.230028 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:21.230190 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:21.231116 (+ 926us) server_negotiation.cc:299] Negotiation successful
0809 06:28:21.231376 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24054,"thread_start_us":24877,"threads_started":1}
I20260809 06:28:21.233129 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:21.181438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41969 (local address 127.8.131.1:32883)
0809 06:28:21.191389 (+ 9951us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:21.191473 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:21.198912 (+ 7439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:21.205726 (+ 6814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:21.205741 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:21.206438 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:21.207400 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:21.207429 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:21.209526 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:21.209534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:21.211747 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:21.211765 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:21.218649 (+ 6884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:21.218657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:21.219011 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:21.219974 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:21.220017 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:21.223163 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:21.228060 (+ 4897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:21.228071 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:21.228090 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:21.228703 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:21.229169 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:21.229174 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:21.229178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:21.229444 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:21.230181 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:21.230189 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:21.230663 (+ 474us) client_negotiation.cc:769] Sending connection context
0809 06:28:21.231019 (+ 356us) client_negotiation.cc:240] Negotiation successful
0809 06:28:21.231596 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9579,"thread_start_us":12721,"threads_started":1}
I20260809 06:28:21.235532 27758 heartbeater.cc:347] Connected to a master server at 127.8.131.62:41969
I20260809 06:28:21.235913 27758 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:21.236828 27758 heartbeater.cc:511] Master 127.8.131.62:41969 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
I20260809 06:28:21.239794 27452 ts_manager.cc:194] Registered new tserver with Master: 2d638e4c7e874680ba2594305c2175a9 (127.8.131.1:38741)
I20260809 06:28:21.241883 27452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:32883
Aug 09 06:28:21 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
W20260809 06:28:21.331509 8716 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.131.2: real 0.165s user 0.003s sys 0.019s
I20260809 06:28:21.332660 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:41969
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:28:21 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
W20260809 06:28:21.838687 27764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:21.839226 27764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:21.839449 27764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:21.851132 27764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:21.851378 27764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:21.851630 27764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:28:21.867439 27764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.131.20:43125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:21.869540 27764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:21.871418 27764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:21.915143 27774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:21.917846 27775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:21.918257 27764 server_base.cc:1034] running on GCE node
W20260809 06:28:21.921916 27780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:21.922914 27764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:21.929818 27764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:21.931114 27764 hybrid_clock.cc:648] HybridClock initialized: now 1786256901931050 us; error 74 us; skew 500 ppm
I20260809 06:28:21.931578 27764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:21 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256901, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:21.936797 27764 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:28:21.952135 27764 webserver.cc:460] Webserver started at http://127.8.131.2:39567/ using document root <none> and password file <none>
I20260809 06:28:21.953368 27764 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:21.953616 27764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:21.954093 27764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 06:28:21 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256901, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:21.987008 27764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "34ea70f0aece4bfab0fbf8e260ea2504"
format_stamp: "Formatted at 2026-08-09 06:28:21 on dist-test-slave-mjjr"
server_key: "5dfb4694f5c6778715cd5206540a83a6"
server_key_iv: "648cd98d660340ee5fd56574f19907ed"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:21.988199 27764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "34ea70f0aece4bfab0fbf8e260ea2504"
format_stamp: "Formatted at 2026-08-09 06:28:21 on dist-test-slave-mjjr"
server_key: "5dfb4694f5c6778715cd5206540a83a6"
server_key_iv: "648cd98d660340ee5fd56574f19907ed"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:21.998624 27764 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.008s sys 0.000s
I20260809 06:28:22.018330 27797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:22.020040 27764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:28:22.020241 27764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "34ea70f0aece4bfab0fbf8e260ea2504"
format_stamp: "Formatted at 2026-08-09 06:28:21 on dist-test-slave-mjjr"
server_key: "5dfb4694f5c6778715cd5206540a83a6"
server_key_iv: "648cd98d660340ee5fd56574f19907ed"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:22.020489 27764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:22.039764 27764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:22.043499 27764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:22.043835 27764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:22.044788 27764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:22.046128 27764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:22.046202 27764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:22.046367 27764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:22.046438 27764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:22.078830 27764 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:39113
I20260809 06:28:22.078851 27955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:39113 every 8 connection(s)
I20260809 06:28:22.080214 27764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:28:22.089243 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 27764
I20260809 06:28:22.089452 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:28:22.090020 8716 external_mini_cluster.cc:1428] Setting key 77d16cbedfec5dad3fe7782c7e20a98c
Aug 09 06:28:22 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256901, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:22.102061 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:22.082840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:49519 (local address 127.8.131.62:41969)
0809 06:28:22.083366 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:22.083373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:22.084435 (+ 1062us) server_negotiation.cc:407] Connection header received
0809 06:28:22.086289 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:22.086294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:22.086376 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:22.086494 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:22.088186 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.089012 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.091256 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.091894 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.092035 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:22.095897 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:22.095902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:22.095906 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:22.095943 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:22.098862 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:22.099603 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:22.099607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:22.099609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:22.099787 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:22.100256 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:22.100261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:22.100265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:22.100555 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:22.100708 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:22.101439 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 06:28:22.101667 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":281,"threads_started":1}
I20260809 06:28:22.103034 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:22.083453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41969 (local address 127.8.131.2:49519)
0809 06:28:22.084271 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:22.084319 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:22.086042 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:22.086706 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:22.086715 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:22.087283 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:22.087987 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:22.088005 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.089184 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.089189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:22.091113 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:22.091129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.092153 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.092163 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:22.092401 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:22.093089 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:22.093111 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:22.095624 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:22.099015 (+ 3391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:22.099024 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:22.099040 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:22.099468 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:22.099906 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:22.099913 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:22.099917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:22.100127 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:22.100699 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:22.100707 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:22.101061 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 06:28:22.101330 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 06:28:22.101868 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":397,"threads_started":1}
I20260809 06:28:22.104490 27956 heartbeater.cc:347] Connected to a master server at 127.8.131.62:41969
I20260809 06:28:22.104801 27956 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:22.105484 27956 heartbeater.cc:511] Master 127.8.131.62:41969 requested a full tablet report, sending...
I20260809 06:28:22.107005 27452 ts_manager.cc:194] Registered new tserver with Master: 34ea70f0aece4bfab0fbf8e260ea2504 (127.8.131.2:39113)
I20260809 06:28:22.108353 27452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:49519
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:28:22.153221 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:41969
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:28:22.245888 27758 heartbeater.cc:503] Master 127.8.131.62:41969 was elected leader, sending a full tablet report...
W20260809 06:28:22.604461 27963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:22.604842 27963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:22.605082 27963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:22.615296 27963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:22.615532 27963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:22.615839 27963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:28:22.629482 27963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.131.20:43125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:22.631188 27963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:22.632823 27963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:22.652109 27970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:22.652208 27969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:22.652174 27972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:22.653064 27963 server_base.cc:1034] running on GCE node
I20260809 06:28:22.653736 27963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:22.654811 27963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:22.656069 27963 hybrid_clock.cc:648] HybridClock initialized: now 1786256902656012 us; error 60 us; skew 500 ppm
I20260809 06:28:22.656563 27963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:22 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256902, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:22.663082 27963 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:28:22.665644 27963 webserver.cc:460] Webserver started at http://127.8.131.3:35591/ using document root <none> and password file <none>
I20260809 06:28:22.666489 27963 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:22.666663 27963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:22.667064 27963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 06:28:22 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256902, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:22.686604 27963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5"
format_stamp: "Formatted at 2026-08-09 06:28:22 on dist-test-slave-mjjr"
server_key: "40639f9c3b91f96840dd90f1abc43a7a"
server_key_iv: "55c10a2044d7cc08fb66d41119371d62"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:22.688055 27963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5"
format_stamp: "Formatted at 2026-08-09 06:28:22 on dist-test-slave-mjjr"
server_key: "40639f9c3b91f96840dd90f1abc43a7a"
server_key_iv: "55c10a2044d7cc08fb66d41119371d62"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:22.694821 27963 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 06:28:22.711591 27979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:22.713275 27963 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 06:28:22.713516 27963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5"
format_stamp: "Formatted at 2026-08-09 06:28:22 on dist-test-slave-mjjr"
server_key: "40639f9c3b91f96840dd90f1abc43a7a"
server_key_iv: "55c10a2044d7cc08fb66d41119371d62"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:22.713821 27963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:22.742352 27963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:22.747504 27963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:22.748032 27963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:22.749446 27963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:22.751518 27963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:22.751725 27963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:22.751941 27963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:22.752054 27963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:22.820796 27963 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:44225
I20260809 06:28:22.820943 28132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:44225 every 8 connection(s)
I20260809 06:28:22.822427 27963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:28:22.824417 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 27963
I20260809 06:28:22.824608 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:28:22.825352 8716 external_mini_cluster.cc:1428] Setting key 6a49b5b611bbd3426af7badb81ee1050
Aug 09 06:28:22 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256902, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:22.849745 28135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:22.826769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:49707 (local address 127.8.131.62:41969)
0809 06:28:22.827842 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:22.827846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:22.829116 (+ 1270us) server_negotiation.cc:407] Connection header received
0809 06:28:22.831716 (+ 2600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:22.831722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:22.831847 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:22.832127 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:22.833999 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.834918 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.836917 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.837566 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.837716 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:22.842604 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:22.842610 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:22.842617 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:22.842774 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:22.846153 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:22.847036 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:22.847042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:22.847047 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:22.847239 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:22.847803 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:22.847807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:22.847809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:22.848122 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:22.848289 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:22.849076 (+ 787us) server_negotiation.cc:299] Negotiation successful
0809 06:28:22.849318 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":724,"threads_started":1}
I20260809 06:28:22.850924 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:22.827334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41969 (local address 127.8.131.3:49707)
0809 06:28:22.828952 (+ 1618us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:22.829011 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:22.831441 (+ 2430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:22.832424 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:22.832433 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:22.833039 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:22.833817 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:22.833836 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.835074 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.835080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:22.836775 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:22.836788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.837922 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.837932 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:22.838197 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:22.838924 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:22.838988 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:22.842383 (+ 3395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:22.846300 (+ 3917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:22.846309 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:22.846324 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:22.846894 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:22.847358 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:22.847364 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:22.847368 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:22.847626 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:22.848267 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:22.848274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:22.848647 (+ 373us) client_negotiation.cc:769] Sending connection context
0809 06:28:22.848954 (+ 307us) client_negotiation.cc:240] Negotiation successful
0809 06:28:22.849551 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1286,"thread_start_us":912,"threads_started":1}
I20260809 06:28:22.852504 28133 heartbeater.cc:347] Connected to a master server at 127.8.131.62:41969
I20260809 06:28:22.852887 28133 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:22.853876 28133 heartbeater.cc:511] Master 127.8.131.62:41969 requested a full tablet report, sending...
I20260809 06:28:22.855723 27452 ts_manager.cc:194] Registered new tserver with Master: f8b0bf4fe7e24d12923262d4e4bc56b5 (127.8.131.3:44225)
I20260809 06:28:22.856683 27452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:49707
I20260809 06:28:22.868083 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:28:22.869938 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 27568
Aug 09 06:28:22 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:28:22.894229 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:22.871426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.8.131.1:38741)
0809 06:28:22.872585 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:22.872592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:22.872616 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:28:22.872685 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:22.872691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:22.872906 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:22.873029 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:22.875256 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.876892 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.879068 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:22.879913 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:22.880011 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:22.885692 (+ 5681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:22.885710 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:22.885731 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:22.885780 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:22.890522 (+ 4742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:22.891372 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:22.891382 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:22.891386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:22.891750 (+ 364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:22.892191 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:22.892196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:22.892199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:22.892633 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:22.893138 (+ 505us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:22.893441 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 06:28:22.893753 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":987,"thread_start_us":677,"threads_started":1}
I20260809 06:28:22.895598 27751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:22.983156 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 27568
I20260809 06:28:23.009492 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 27764
Aug 09 06:28:23 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:28:23.031404 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:23.010480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.8.131.2:39113)
0809 06:28:23.011038 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:23.011045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:23.011071 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:28:23.011133 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:23.011139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:23.011361 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:23.011540 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:23.012927 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:23.013961 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:23.015363 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:23.016089 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:23.016240 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:23.025444 (+ 9204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:23.025457 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:23.025472 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:23.025511 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:23.028420 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:23.029141 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:23.029149 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:23.029151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:23.029287 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:23.029684 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:23.029689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:23.029694 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:23.030066 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:23.030407 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:23.030738 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 06:28:23.031030 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":280,"threads_started":1}
I20260809 06:28:23.032745 27951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:23.114936 27956 heartbeater.cc:503] Master 127.8.131.62:41969 was elected leader, sending a full tablet report...
I20260809 06:28:23.116462 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 27764
W20260809 06:28:23.136772 27427 connection.cc:535] server connection from 127.8.131.2:49519 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 06:28:23.137041 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 27963
Aug 09 06:28:23 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:28:23.153338 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:23.138150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45312 (local address 127.8.131.3:44225)
0809 06:28:23.138830 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:23.138837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:23.138868 (+ 31us) server_negotiation.cc:407] Connection header received
0809 06:28:23.138938 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:23.138945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:23.139152 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:23.139285 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:23.140546 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:23.141565 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:23.142874 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:23.143437 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:23.143581 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:23.147961 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:23.147972 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:23.147985 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:23.148016 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:23.150818 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:23.151423 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:23.151430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:23.151432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:23.151557 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:23.151927 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:23.151930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:23.151932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:23.152268 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:23.152514 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:23.152758 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 06:28:23.152948 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":374,"threads_started":1}
I20260809 06:28:23.154686 28128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:23.239051 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 27963
I20260809 06:28:23.261169 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 27403
I20260809 06:28:23.262492 27497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:23.370213 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 27403
I20260809 06:28:23.398576 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41969
--webserver_interface=127.8.131.62
--webserver_port=43837
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:41969
--ranger_config_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43125/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.131.20:43125/kms with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:23.732457 28147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:23.732832 28147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:23.733012 28147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:23.740489 28147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:28:23.740602 28147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:23.740737 28147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:23.740842 28147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:28:23.740935 28147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:28:23.751256 28147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.131.20:43125/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:41969
--ranger_config_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=43837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:23.752651 28147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:23.754047 28147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:23.766753 28156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:23.766791 28158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:23.766817 28153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:23.767544 28147 server_base.cc:1034] running on GCE node
I20260809 06:28:23.768374 28147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:23.769771 28147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:23.770996 28147 hybrid_clock.cc:648] HybridClock initialized: now 1786256903770940 us; error 46 us; skew 500 ppm
I20260809 06:28:23.771402 28147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:23 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256903, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:23.774724 28147 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:28:23.776304 28147 webserver.cc:460] Webserver started at http://127.8.131.62:43837/ using document root <none> and password file <none>
I20260809 06:28:23.776993 28147 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:23.777133 28147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 06:28:23 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256903, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:23.827239 28147 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 06:28:23.830214 28167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:23.831558 28147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:28:23.831799 28147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "af46194188084aef95fe79109e1ec8eb"
format_stamp: "Formatted at 2026-08-09 06:28:19 on dist-test-slave-mjjr"
server_key: "1be4b1312fb1d35f40bd188e3fe0e42d"
server_key_iv: "3102de1ec2b6f47a99a839c48373ebaa"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:23.832791 28147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:23.849577 28147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:23.853562 28147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:23.853852 28147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:23.873805 28147 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:41969
I20260809 06:28:23.873829 28243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:41969 every 8 connection(s)
I20260809 06:28:23.875510 28147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:28:23.882470 28245 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:28:23.884961 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 28147
I20260809 06:28:23.885767 28245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Bootstrap starting.
I20260809 06:28:23.885815 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:38741
--local_ip_for_outbound_sockets=127.8.131.1
--tserver_master_addrs=127.8.131.62:41969
--webserver_port=46551
--webserver_interface=127.8.131.1
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.131.20:43125/kms with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:28:23.904243 28245 log.cc:824] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:23.913110 28245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:28:23.913646 28245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Bootstrap complete.
I20260809 06:28:23.922608 28245 raft_consensus.cc:348] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } }
I20260809 06:28:23.923488 28245 raft_consensus.cc:729] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: af46194188084aef95fe79109e1ec8eb, State: Initialized, Role: FOLLOWER
I20260809 06:28:23.924182 28245 consensus_queue.cc:260] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } }
I20260809 06:28:23.924434 28245 raft_consensus.cc:388] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:28:23.924562 28245 raft_consensus.cc:482] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:28:23.924759 28245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:28:23.928550 28245 raft_consensus.cc:504] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } }
I20260809 06:28:23.929047 28245 leader_election.cc:302] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af46194188084aef95fe79109e1ec8eb; no voters:
I20260809 06:28:23.929688 28245 leader_election.cc:288] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 06:28:23.929831 28250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 06:28:23.931103 28250 raft_consensus.cc:686] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [term 2 LEADER]: Becoming Leader. State: Replica: af46194188084aef95fe79109e1ec8eb, State: Running, Role: LEADER
I20260809 06:28:23.931715 28250 consensus_queue.cc:237] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } }
I20260809 06:28:23.931945 28245 sys_catalog.cc:564] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:28:23.934430 28252 sys_catalog.cc:455] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "af46194188084aef95fe79109e1ec8eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } } }
I20260809 06:28:23.934862 28252 sys_catalog.cc:458] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:23.938759 28253 sys_catalog.cc:455] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader af46194188084aef95fe79109e1ec8eb. Latest consensus state: current_term: 2 leader_uuid: "af46194188084aef95fe79109e1ec8eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af46194188084aef95fe79109e1ec8eb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41969 } } }
I20260809 06:28:23.939095 28253 sys_catalog.cc:458] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:23.940877 28258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:28:23.946318 28258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:28:23.947741 28258 catalog_manager.cc:1259] Loaded cluster ID: 650679e286ee4e2c9886561274344316
I20260809 06:28:23.947832 28258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:28:23.951721 28258 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:28:23.953344 28258 catalog_manager.cc:5879] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Loaded TSK: 0
I20260809 06:28:23.954818 28258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:28:23.962664 28245 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 06:28:24.323235 28247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:24.323632 28247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:24.323827 28247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:24.335438 28247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:24.335587 28247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:24.335793 28247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:28:24.350931 28247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.131.20:43125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:38741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=46551
--tserver_master_addrs=127.8.131.62:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:24.352666 28247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:24.354377 28247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:24.376560 28297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:24.379613 28298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:24.390300 28302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:24.391150 28247 server_base.cc:1034] running on GCE node
I20260809 06:28:24.392019 28247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:24.393395 28247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:24.394645 28247 hybrid_clock.cc:648] HybridClock initialized: now 1786256904394571 us; error 56 us; skew 500 ppm
I20260809 06:28:24.395002 28247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:24 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256904, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:24.398963 28247 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:28:24.400861 28247 webserver.cc:460] Webserver started at http://127.8.131.1:46551/ using document root <none> and password file <none>
I20260809 06:28:24.401587 28247 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:24.401754 28247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 06:28:24 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256904, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:24.427223 28247 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 06:28:24.430444 28310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:24.432178 28247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 06:28:24.432432 28247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "2d638e4c7e874680ba2594305c2175a9"
format_stamp: "Formatted at 2026-08-09 06:28:20 on dist-test-slave-mjjr"
server_key: "ea975348c704713d82b0f651375c42da"
server_key_iv: "c63f22c21db6e9e1d0cf477310bad1ec"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:24.434276 28247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:24.499027 28247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:24.504554 28247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:24.504899 28247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:24.512126 28247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:24.520041 28247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:24.520183 28247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:24.520310 28247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:24.520394 28247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:24.617236 28247 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:38741
I20260809 06:28:24.617286 28467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:38741 every 8 connection(s)
I20260809 06:28:24.619318 28247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:28:24.627296 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 28247
I20260809 06:28:24.628132 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:39113
--local_ip_for_outbound_sockets=127.8.131.2
--tserver_master_addrs=127.8.131.62:41969
--webserver_port=39567
--webserver_interface=127.8.131.2
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.131.20:43125/kms with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:28:24 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256904, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:24.685065 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:24.623393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41969 (local address 127.8.131.1:59713)
0809 06:28:24.626559 (+ 3166us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:24.636385 (+ 9826us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:24.639417 (+ 3032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:24.642006 (+ 2589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:24.642020 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:24.642759 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:24.643558 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:24.643586 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:24.646330 (+ 2744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:24.646339 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:24.648654 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:24.648672 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:24.652341 (+ 3669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:24.652352 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:24.652660 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:24.654247 (+ 1587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:24.654282 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:24.674403 (+ 20121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:24.679485 (+ 5082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:24.679495 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:24.679513 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:24.680241 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:24.680816 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:24.680823 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:24.680828 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:24.681101 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:24.682300 (+ 1199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:24.682308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:24.682708 (+ 400us) client_negotiation.cc:769] Sending connection context
0809 06:28:24.683036 (+ 328us) client_negotiation.cc:240] Negotiation successful
0809 06:28:24.683624 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2817,"thread_start_us":692,"threads_started":1}
I20260809 06:28:24.687626 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:24.624852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:59713 (local address 127.8.131.62:41969)
0809 06:28:24.626385 (+ 1533us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:24.626404 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:24.639909 (+ 13505us) server_negotiation.cc:407] Connection header received
0809 06:28:24.640365 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:24.640386 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:24.640886 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:24.641298 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:24.643847 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:24.646122 (+ 2275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:24.648813 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:24.651627 (+ 2814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:24.651875 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:24.674858 (+ 22983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:24.674873 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:24.674895 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:24.674977 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:24.679265 (+ 4288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:24.680442 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:24.680454 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:24.680461 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:24.680668 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:24.681250 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:24.681255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:24.681259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:24.682002 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:24.682216 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:24.685963 (+ 3747us) server_negotiation.cc:299] Negotiation successful
0809 06:28:24.686383 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1283,"thread_start_us":1016,"threads_started":1}
I20260809 06:28:24.690701 28468 heartbeater.cc:347] Connected to a master server at 127.8.131.62:41969
I20260809 06:28:24.691051 28468 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:24.692011 28468 heartbeater.cc:511] Master 127.8.131.62:41969 requested a full tablet report, sending...
W20260809 06:28:24.695331 28468 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:41969 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 06:28:24.695591 28468 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:24.695904 28468 heartbeater.cc:511] Master 127.8.131.62:41969 requested a full tablet report, sending...
I20260809 06:28:24.697086 28468 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:24.697355 28468 heartbeater.cc:511] Master 127.8.131.62:41969 requested a full tablet report, sending...
W20260809 06:28:24.698405 28468 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 06:28:25.089576 28474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:25.089944 28474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:25.090152 28474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:25.101804 28474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:25.101943 28474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:25.102186 28474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:28:25.115065 28474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.131.20:43125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:39113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=39567
--tserver_master_addrs=127.8.131.62:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:25.116549 28474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:25.118007 28474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:25.147821 28496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:25.152490 28495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:25.155925 28474 server_base.cc:1034] running on GCE node
W20260809 06:28:25.156785 28500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:25.157702 28474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:25.159173 28474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:25.162688 28474 hybrid_clock.cc:648] HybridClock initialized: now 1786256905162598 us; error 67 us; skew 500 ppm
I20260809 06:28:25.163143 28474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:25 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256905, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:25.170727 28474 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:28:25.175771 28474 webserver.cc:460] Webserver started at http://127.8.131.2:39567/ using document root <none> and password file <none>
I20260809 06:28:25.176808 28474 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:25.177032 28474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 06:28:25 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256905, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:25.213785 28474 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.001s
I20260809 06:28:25.232389 28514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:25.238070 28474 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.005s sys 0.012s
I20260809 06:28:25.238381 28474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "34ea70f0aece4bfab0fbf8e260ea2504"
format_stamp: "Formatted at 2026-08-09 06:28:21 on dist-test-slave-mjjr"
server_key: "5dfb4694f5c6778715cd5206540a83a6"
server_key_iv: "648cd98d660340ee5fd56574f19907ed"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:25.240092 28474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 06:28:25 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786256905, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:25.312829 28474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:25.317719 28474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:25.318041 28474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:25.319095 28474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:25.320688 28474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:25.320782 28474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:25.320958 28474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:25.321064 28474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:25.383671 28474 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:39113
I20260809 06:28:25.383730 28673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:39113 every 8 connection(s)
I20260809 06:28:25.385421 28474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:28:25.387042 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 28474
I20260809 06:28:25.388144 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.131.20:43125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:44225
--local_ip_for_outbound_sockets=127.8.131.3
--tserver_master_addrs=127.8.131.62:41969
--webserver_port=35591
--webserver_interface=127.8.131.3
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.131.20:43125/kms with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:28:25 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256905, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:25.418851 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:25.394747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:34235 (local address 127.8.131.62:41969)
0809 06:28:25.400433 (+ 5686us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:25.400441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:25.400466 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:28:25.400537 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:25.400543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:25.400631 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:25.400740 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:25.405660 (+ 4920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:25.406387 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:25.408100 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:25.408537 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:25.408695 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:25.412211 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:25.412216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:25.412220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:25.412255 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:25.415107 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:25.416057 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:25.416063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:25.416066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:25.416212 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:25.416767 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:25.416772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:25.416776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:25.417144 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:25.417298 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:25.418214 (+ 916us) server_negotiation.cc:299] Negotiation successful
0809 06:28:25.418465 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5522,"thread_start_us":6277,"threads_started":1}
I20260809 06:28:25.420068 28679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:25.394106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41969 (local address 127.8.131.2:34235)
0809 06:28:25.396573 (+ 2467us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:25.396630 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:25.399346 (+ 2716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:25.404093 (+ 4747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:25.404105 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:25.404754 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:25.405464 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:25.405483 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:25.406595 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:25.406600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:25.407952 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:25.407965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:25.408790 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:25.408797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:25.409027 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:25.409753 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:25.409782 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:25.411975 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:25.415247 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:25.415256 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:25.415274 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:25.415895 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:25.416347 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:25.416352 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:25.416356 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:25.416625 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:25.417373 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:25.417381 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:25.417794 (+ 413us) client_negotiation.cc:769] Sending connection context
0809 06:28:25.418103 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 06:28:25.418657 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2120,"thread_start_us":1899,"threads_started":1}
I20260809 06:28:25.421582 28675 heartbeater.cc:347] Connected to a master server at 127.8.131.62:41969
I20260809 06:28:25.421916 28675 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:25.422608 28675 heartbeater.cc:511] Master 127.8.131.62:41969 requested a full tablet report, sending...
I20260809 06:28:25.424340 28191 ts_manager.cc:194] Registered new tserver with Master: 34ea70f0aece4bfab0fbf8e260ea2504 (127.8.131.2:39113)
I20260809 06:28:25.425925 28191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:34235
I20260809 06:28:25.698589 28468 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:25.699354 28468 heartbeater.cc:511] Master 127.8.131.62:41969 requested a full tablet report, sending...
I20260809 06:28:25.701174 28191 ts_manager.cc:194] Registered new tserver with Master: 2d638e4c7e874680ba2594305c2175a9 (127.8.131.1:38741)
I20260809 06:28:25.702526 28191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:59713
Aug 09 06:28:25 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256905, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
W20260809 06:28:25.866849 28681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:25.867456 28681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:25.867800 28681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:25.882825 28681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:25.883180 28681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:25.883558 28681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:28:25.912513 28681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.131.20:43125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:44225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=35591
--tserver_master_addrs=127.8.131.62:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:25.914109 28681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:25.915797 28681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:25.938611 28691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:25.939040 28690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:25.942171 28693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:25.942628 28681 server_base.cc:1034] running on GCE node
I20260809 06:28:25.943290 28681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:25.945276 28681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:25.946650 28681 hybrid_clock.cc:648] HybridClock initialized: now 1786256905946461 us; error 101 us; skew 500 ppm
I20260809 06:28:25.947696 28681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:25 dist-test-slave-mjjr krb5kdc[26801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256905, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:25.953915 28681 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:28:25.957634 28681 webserver.cc:460] Webserver started at http://127.8.131.3:35591/ using document root <none> and password file <none>
I20260809 06:28:25.958918 28681 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:25.959244 28681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 06:28:25 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256905, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:25.994066 28681 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.004s
I20260809 06:28:25.999158 28703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:26.001362 28681 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260809 06:28:26.001614 28681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5"
format_stamp: "Formatted at 2026-08-09 06:28:22 on dist-test-slave-mjjr"
server_key: "40639f9c3b91f96840dd90f1abc43a7a"
server_key_iv: "55c10a2044d7cc08fb66d41119371d62"
server_key_version: "kuduclusterkey@0"
I20260809 06:28:26.003470 28681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:26.046057 28681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:26.054013 28681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:26.054570 28681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:26.056726 28681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:26.059703 28681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:26.060019 28681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 06:28:26.060298 28681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:26.060559 28681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 06:28:26 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256905, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for HTTP/127.8.131.20@KRBTEST.COM
I20260809 06:28:26.209549 28681 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:44225
I20260809 06:28:26.211709 28681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:28:26.209591 28856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:44225 every 8 connection(s)
I20260809 06:28:26.213135 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 28681
Aug 09 06:28:26 dist-test-slave-mjjr krb5kdc[26801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256905, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:26.263983 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.214269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37698 (local address 127.8.131.62:41969)
0809 06:28:26.216339 (+ 2070us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.216346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.218383 (+ 2037us) server_negotiation.cc:407] Connection header received
0809 06:28:26.218618 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.218628 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.218767 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.218895 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:26.245431 (+ 26536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.246636 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.248927 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.249600 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.249734 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.256870 (+ 7136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:26.256877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:26.256881 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:26.256930 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:26.260578 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:26.261452 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:26.261458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:26.261461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:26.261642 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:26.262122 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:26.262128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:26.262131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:26.262497 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:26.262741 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.263350 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.263581 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1943,"thread_start_us":1783,"threads_started":1}
I20260809 06:28:26.266280 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.222914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41969 (local address 127.8.131.3:39703)
0809 06:28:26.224052 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:26.224098 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:26.228658 (+ 4560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:26.239100 (+ 10442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:26.239113 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:26.239735 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:26.240657 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.240686 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.244446 (+ 3760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.244453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.246630 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.246650 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.249114 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.249121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.249437 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.250381 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:26.250413 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:26.253591 (+ 3178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:26.261461 (+ 7870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:26.261471 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:26.261488 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:26.262119 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:26.262526 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:26.262534 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:26.262539 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:26.262801 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:26.263467 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:26.263474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:26.263897 (+ 423us) client_negotiation.cc:769] Sending connection context
0809 06:28:26.264212 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 06:28:26.264876 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":617,"threads_started":1}
I20260809 06:28:26.268078 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.223488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:39703 (local address 127.8.131.62:41969)
0809 06:28:26.236577 (+ 13089us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.236583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.236605 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:28:26.236670 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.236676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.236777 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.239746 (+ 2969us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:26.243329 (+ 3583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.244273 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.246769 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.247360 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.249853 (+ 2493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.253837 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:26.253844 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:26.253848 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:26.253899 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:26.261314 (+ 7415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:26.262265 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:26.262269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:26.262272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:26.262394 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:26.262969 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:26.262974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:26.262979 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:26.263318 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:26.267344 (+ 4026us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.267467 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.267701 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12947,"spinlock_wait_cycles":1280,"thread_start_us":14015,"threads_started":1}
I20260809 06:28:26.269018 28857 heartbeater.cc:347] Connected to a master server at 127.8.131.62:41969
I20260809 06:28:26.269327 28857 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:26.270011 28857 heartbeater.cc:511] Master 127.8.131.62:41969 requested a full tablet report, sending...
I20260809 06:28:26.271373 28191 ts_manager.cc:194] Registered new tserver with Master: f8b0bf4fe7e24d12923262d4e4bc56b5 (127.8.131.3:44225)
I20260809 06:28:26.272511 28191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:39703
I20260809 06:28:26.280268 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:28:26.310796 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.284194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37704 (local address 127.8.131.62:41969)
0809 06:28:26.284415 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.284421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.291297 (+ 6876us) server_negotiation.cc:407] Connection header received
0809 06:28:26.291904 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.291911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.292019 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.292133 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:26.294452 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.295609 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.297824 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.298552 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.298675 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.302763 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:26.302771 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:26.302777 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:26.302826 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:26.306815 (+ 3989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:26.307758 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:26.307764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:26.307768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:26.307952 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:26.308457 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:26.308464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:26.308468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:26.308909 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:26.309183 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.309963 (+ 780us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.310394 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 06:28:26.318785 28191 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:28:26.331465 28191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:28:26.406852 28879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.363699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:44225 (local address 127.0.0.1:37324)
0809 06:28:26.394811 (+ 31112us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:26.394831 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:26.394965 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:26.398038 (+ 3073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:26.398044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:26.398073 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:26.398447 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.398458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.401272 (+ 2814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.401277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.403299 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.403314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.405794 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.405802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.406031 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.406088 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 06:28:26.406234 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 06:28:26.406435 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30977,"thread_start_us":10701,"threads_started":1}
I20260809 06:28:26.407748 28884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.364850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37324 (local address 127.8.131.3:44225)
0809 06:28:26.373133 (+ 8283us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.373142 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.395110 (+ 21968us) server_negotiation.cc:407] Connection header received
0809 06:28:26.395224 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.395232 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.395506 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.395751 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:26.398658 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.400409 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.403501 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.405217 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.405364 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.405613 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.407122 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.407311 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8110,"thread_start_us":10220,"threads_started":1}
I20260809 06:28:26.408571 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.364458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.8.131.2:39113)
0809 06:28:26.372641 (+ 8183us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.372652 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.374840 (+ 2188us) server_negotiation.cc:407] Connection header received
0809 06:28:26.374965 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.374973 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.375216 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.375422 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:26.389845 (+ 14423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.391588 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.396119 (+ 4531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.397595 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.397720 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.397958 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.408047 (+ 10089us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.408217 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8013,"thread_start_us":7869,"threads_started":1}
I20260809 06:28:26.411273 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.363354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:39113 (local address 127.0.0.1:59304)
0809 06:28:26.374358 (+ 11004us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:26.374412 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:26.374602 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:26.389126 (+ 14524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:26.389135 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:26.389190 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:26.389591 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.389606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.391790 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.391796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.394273 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.394295 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.407274 (+ 12979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.407281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.407460 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.407494 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:28:26.407583 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 06:28:26.407733 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10723,"thread_start_us":37672,"threads_started":1}
I20260809 06:28:26.414677 28770 tablet_service.cc:1467] Processing CreateTablet for tablet 02d5e7ac5285463f853677690a2fd801 (DEFAULT_TABLE table=test-table [id=ec145db8fa904ec588ea2185f33605da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:28:26.415236 28577 tablet_service.cc:1467] Processing CreateTablet for tablet 02d5e7ac5285463f853677690a2fd801 (DEFAULT_TABLE table=test-table [id=ec145db8fa904ec588ea2185f33605da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:28:26.416369 28770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02d5e7ac5285463f853677690a2fd801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:26.416610 28577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02d5e7ac5285463f853677690a2fd801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:26.426960 28886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.376094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38741 (local address 127.0.0.1:53414)
0809 06:28:26.400959 (+ 24865us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:26.400982 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:26.403666 (+ 2684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:26.404603 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:26.404608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:26.404634 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:26.404996 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.405007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.408821 (+ 3814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.408827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.410851 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.410878 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.426087 (+ 15209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.426094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.426283 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.426318 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 06:28:26.426426 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 06:28:26.426604 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24715,"thread_start_us":25461,"threads_started":1}
I20260809 06:28:26.427131 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.376403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.8.131.1:38741)
0809 06:28:26.388822 (+ 12419us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.388832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.403532 (+ 14700us) server_negotiation.cc:407] Connection header received
0809 06:28:26.403889 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.403899 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.404143 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.404316 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:26.405211 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.406838 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.423922 (+ 17084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.425518 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.425631 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.425862 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.426546 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.426749 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12229,"thread_start_us":12444,"threads_started":1}
I20260809 06:28:26.430941 28675 heartbeater.cc:503] Master 127.8.131.62:41969 was elected leader, sending a full tablet report...
I20260809 06:28:26.432418 28376 tablet_service.cc:1467] Processing CreateTablet for tablet 02d5e7ac5285463f853677690a2fd801 (DEFAULT_TABLE table=test-table [id=ec145db8fa904ec588ea2185f33605da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:28:26.433981 28376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02d5e7ac5285463f853677690a2fd801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:26.444499 28899 tablet_bootstrap.cc:492] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: Bootstrap starting.
I20260809 06:28:26.448247 28899 tablet_bootstrap.cc:654] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:26.449679 28899 log.cc:824] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:26.450608 28900 tablet_bootstrap.cc:492] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: Bootstrap starting.
I20260809 06:28:26.454572 28900 tablet_bootstrap.cc:654] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:26.455507 28901 tablet_bootstrap.cc:492] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: Bootstrap starting.
I20260809 06:28:26.456089 28900 log.cc:824] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:26.459224 28901 tablet_bootstrap.cc:654] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:26.459506 28900 tablet_bootstrap.cc:492] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: No bootstrap required, opened a new log
I20260809 06:28:26.459784 28899 tablet_bootstrap.cc:492] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: No bootstrap required, opened a new log
I20260809 06:28:26.459825 28900 ts_tablet_manager.cc:1397] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 06:28:26.460072 28899 ts_tablet_manager.cc:1397] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.009s
I20260809 06:28:26.460529 28901 log.cc:824] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:26.464165 28901 tablet_bootstrap.cc:492] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: No bootstrap required, opened a new log
I20260809 06:28:26.464454 28901 ts_tablet_manager.cc:1397] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 06:28:26.471266 28899 raft_consensus.cc:348] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } }
I20260809 06:28:26.471266 28900 raft_consensus.cc:348] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } }
I20260809 06:28:26.471757 28900 raft_consensus.cc:374] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:26.471920 28899 raft_consensus.cc:374] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:26.471906 28900 raft_consensus.cc:729] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ea70f0aece4bfab0fbf8e260ea2504, State: Initialized, Role: FOLLOWER
I20260809 06:28:26.472054 28899 raft_consensus.cc:729] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8b0bf4fe7e24d12923262d4e4bc56b5, State: Initialized, Role: FOLLOWER
I20260809 06:28:26.472699 28900 consensus_queue.cc:260] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } }
I20260809 06:28:26.472699 28899 consensus_queue.cc:260] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } }
I20260809 06:28:26.472770 28901 raft_consensus.cc:348] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } }
I20260809 06:28:26.473352 28901 raft_consensus.cc:374] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:26.473537 28901 raft_consensus.cc:729] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d638e4c7e874680ba2594305c2175a9, State: Initialized, Role: FOLLOWER
I20260809 06:28:26.474107 28857 heartbeater.cc:503] Master 127.8.131.62:41969 was elected leader, sending a full tablet report...
I20260809 06:28:26.474277 28901 consensus_queue.cc:260] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } }
I20260809 06:28:26.474967 28899 ts_tablet_manager.cc:1428] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260809 06:28:26.476673 28468 heartbeater.cc:503] Master 127.8.131.62:41969 was elected leader, sending a full tablet report...
I20260809 06:28:26.477327 28901 ts_tablet_manager.cc:1428] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260809 06:28:26.478448 28900 ts_tablet_manager.cc:1428] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: Time spent starting tablet: real 0.019s user 0.014s sys 0.000s
W20260809 06:28:26.487847 28864 tablet.cc:2369] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:28:26.630178 28470 tablet.cc:2369] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:28:26.652801 28676 tablet.cc:2369] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:28:26.762049 28905 raft_consensus.cc:482] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:28:26.762326 28905 raft_consensus.cc:504] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } }
I20260809 06:28:26.764146 28905 leader_election.cc:288] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8b0bf4fe7e24d12923262d4e4bc56b5 (127.8.131.3:44225), 2d638e4c7e874680ba2594305c2175a9 (127.8.131.1:38741)
I20260809 06:28:26.771431 28909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.764532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:44225 (local address 127.8.131.2:52729)
0809 06:28:26.765251 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:26.765278 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:26.765431 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:26.766013 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:26.766017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:26.766042 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:26.766457 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.766484 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.767744 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.767749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.769434 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.769445 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.770605 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.770611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.770766 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.770789 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 06:28:26.770948 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 06:28:26.771080 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":412,"threads_started":1}
I20260809 06:28:26.771440 28884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.764619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:52729 (local address 127.8.131.3:44225)
0809 06:28:26.764853 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.764858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.765370 (+ 512us) server_negotiation.cc:407] Connection header received
0809 06:28:26.765663 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.765669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.765751 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.765869 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:26.766628 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.767582 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.769592 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.770364 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.770514 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.770639 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.771013 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.771166 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 06:28:26.772593 28794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02d5e7ac5285463f853677690a2fd801" candidate_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" is_pre_election: true
I20260809 06:28:26.773041 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.765731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:53323 (local address 127.8.131.1:38741)
0809 06:28:26.765967 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.765974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.766170 (+ 196us) server_negotiation.cc:407] Connection header received
0809 06:28:26.766349 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.766354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.766449 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.766572 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:26.767156 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.768284 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.770907 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.772119 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.772261 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.772371 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.772658 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.772778 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 06:28:26.773113 28910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.765421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38741 (local address 127.8.131.2:53323)
0809 06:28:26.766055 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:26.766072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:26.766213 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:26.766608 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:26.766612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:26.766635 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:26.767002 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.767014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.768440 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.768446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.770722 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:26.770737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.772333 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.772339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:26.772496 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.772519 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 06:28:26.772622 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 06:28:26.772740 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":4,"thread_start_us":547,"threads_started":1}
I20260809 06:28:26.773125 28794 raft_consensus.cc:2454] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34ea70f0aece4bfab0fbf8e260ea2504 in term 0.
I20260809 06:28:26.773818 28517 leader_election.cc:302] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34ea70f0aece4bfab0fbf8e260ea2504, f8b0bf4fe7e24d12923262d4e4bc56b5; no voters:
I20260809 06:28:26.774281 28905 raft_consensus.cc:2781] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:28:26.774124 28402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02d5e7ac5285463f853677690a2fd801" candidate_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d638e4c7e874680ba2594305c2175a9" is_pre_election: true
I20260809 06:28:26.774430 28905 raft_consensus.cc:482] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:28:26.774570 28905 raft_consensus.cc:3037] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:26.774664 28402 raft_consensus.cc:2454] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34ea70f0aece4bfab0fbf8e260ea2504 in term 0.
I20260809 06:28:26.777045 28905 raft_consensus.cc:504] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } }
I20260809 06:28:26.777621 28905 leader_election.cc:288] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [CANDIDATE]: Term 1 election: Requested vote from peers f8b0bf4fe7e24d12923262d4e4bc56b5 (127.8.131.3:44225), 2d638e4c7e874680ba2594305c2175a9 (127.8.131.1:38741)
I20260809 06:28:26.778028 28794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02d5e7ac5285463f853677690a2fd801" candidate_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5"
I20260809 06:28:26.778113 28402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02d5e7ac5285463f853677690a2fd801" candidate_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d638e4c7e874680ba2594305c2175a9"
I20260809 06:28:26.778215 28794 raft_consensus.cc:3037] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:26.778355 28402 raft_consensus.cc:3037] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:26.780395 28794 raft_consensus.cc:2454] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34ea70f0aece4bfab0fbf8e260ea2504 in term 1.
I20260809 06:28:26.780840 28517 leader_election.cc:302] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34ea70f0aece4bfab0fbf8e260ea2504, f8b0bf4fe7e24d12923262d4e4bc56b5; no voters:
I20260809 06:28:26.780951 28402 raft_consensus.cc:2454] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34ea70f0aece4bfab0fbf8e260ea2504 in term 1.
I20260809 06:28:26.781102 28905 raft_consensus.cc:2781] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:28:26.781711 28905 raft_consensus.cc:686] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 1 LEADER]: Becoming Leader. State: Replica: 34ea70f0aece4bfab0fbf8e260ea2504, State: Running, Role: LEADER
I20260809 06:28:26.782264 28905 consensus_queue.cc:237] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } }
I20260809 06:28:26.786583 28190 catalog_manager.cc:5495] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34ea70f0aece4bfab0fbf8e260ea2504 (127.8.131.2). New cstate: current_term: 1 leader_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34ea70f0aece4bfab0fbf8e260ea2504" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 } health_report { overall_health: UNKNOWN } } }
I20260809 06:28:26.839766 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.833711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.8.131.2:39113)
0809 06:28:26.834042 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.834065 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.834321 (+ 256us) server_negotiation.cc:407] Connection header received
0809 06:28:26.834667 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.834675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.834816 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.834973 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:28:26.835356 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.836242 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.837688 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.838081 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.838175 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.838615 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:28:26.839152 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:28:26.839293 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.839359 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.839450 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 06:28:26.848306 28794 raft_consensus.cc:1261] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [term 1 FOLLOWER]: Refusing update from remote peer 34ea70f0aece4bfab0fbf8e260ea2504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:28:26.848308 28402 raft_consensus.cc:1261] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [term 1 FOLLOWER]: Refusing update from remote peer 34ea70f0aece4bfab0fbf8e260ea2504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:28:26.849278 28911 consensus_queue.cc:1035] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8b0bf4fe7e24d12923262d4e4bc56b5" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 44225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:28:26.849655 28905 consensus_queue.cc:1035] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d638e4c7e874680ba2594305c2175a9" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:28:26.873381 28190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:28:26.873693 28190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:28:26.877660 28190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af46194188084aef95fe79109e1ec8eb: Sending DeleteTablet for 3 replicas of tablet 02d5e7ac5285463f853677690a2fd801
I20260809 06:28:26.879001 28770 tablet_service.cc:1514] Processing DeleteTablet for tablet 02d5e7ac5285463f853677690a2fd801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:28:26 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:37324
I20260809 06:28:26.879127 28376 tablet_service.cc:1514] Processing DeleteTablet for tablet 02d5e7ac5285463f853677690a2fd801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:28:26 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:53414
I20260809 06:28:26.879261 28577 tablet_service.cc:1514] Processing DeleteTablet for tablet 02d5e7ac5285463f853677690a2fd801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:28:26 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:59304
I20260809 06:28:26.891034 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 28247
I20260809 06:28:26.920351 28926 tablet_replica.cc:330] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: stopping tablet replica
I20260809 06:28:26.921264 28926 raft_consensus.cc:2229] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:28:26.926838 28926 raft_consensus.cc:2258] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:28:26.927672 28927 tablet_replica.cc:330] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: stopping tablet replica
I20260809 06:28:26.928234 28927 raft_consensus.cc:2229] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:28:26.928838 28927 raft_consensus.cc:2258] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:28:26.930027 28929 tablet_replica.cc:330] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: stopping tablet replica
I20260809 06:28:26.930511 28929 raft_consensus.cc:2229] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:28:26.931269 28929 raft_consensus.cc:2258] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:28:26.933878 28929 ts_tablet_manager.cc:1905] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:28:26.934015 28926 ts_tablet_manager.cc:1905] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:28:26.937300 28927 ts_tablet_manager.cc:1905] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:28:26.942708 28929 ts_tablet_manager.cc:1918] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:28:26.942998 28929 log.cc:1196] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/02d5e7ac5285463f853677690a2fd801
I20260809 06:28:26.943728 28929 ts_tablet_manager.cc:1939] T 02d5e7ac5285463f853677690a2fd801 P 34ea70f0aece4bfab0fbf8e260ea2504: Deleting consensus metadata
I20260809 06:28:26.945792 28169 catalog_manager.cc:4841] TS 34ea70f0aece4bfab0fbf8e260ea2504 (127.8.131.2:39113): tablet 02d5e7ac5285463f853677690a2fd801 (table test-table [id=ec145db8fa904ec588ea2185f33605da]) successfully deleted
I20260809 06:28:26.946448 28926 ts_tablet_manager.cc:1918] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:28:26.946655 28926 log.cc:1196] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/02d5e7ac5285463f853677690a2fd801
I20260809 06:28:26.947228 28926 ts_tablet_manager.cc:1939] T 02d5e7ac5285463f853677690a2fd801 P f8b0bf4fe7e24d12923262d4e4bc56b5: Deleting consensus metadata
I20260809 06:28:26.949270 28168 catalog_manager.cc:4841] TS f8b0bf4fe7e24d12923262d4e4bc56b5 (127.8.131.3:44225): tablet 02d5e7ac5285463f853677690a2fd801 (table test-table [id=ec145db8fa904ec588ea2185f33605da]) successfully deleted
I20260809 06:28:26.953455 28927 ts_tablet_manager.cc:1918] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:28:26.953679 28927 log.cc:1196] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/02d5e7ac5285463f853677690a2fd801
I20260809 06:28:26.953914 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:26.892336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.8.131.1:38741)
0809 06:28:26.892539 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:26.892545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:26.911116 (+ 18571us) server_negotiation.cc:407] Connection header received
0809 06:28:26.911808 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:26.911817 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:26.911976 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:26.912075 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:26.921694 (+ 9619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.923245 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.929877 (+ 6632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:26.930565 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:26.930687 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:26.945667 (+ 14980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:26.945683 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:26.945702 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:26.945758 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:26.949998 (+ 4240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:26.951013 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:26.951022 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:26.951026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:26.951206 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:26.951722 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:26.951728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:26.951733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:26.952221 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:26.952578 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:26.952907 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 06:28:26.953153 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 06:28:26.954329 28927 ts_tablet_manager.cc:1939] T 02d5e7ac5285463f853677690a2fd801 P 2d638e4c7e874680ba2594305c2175a9: Deleting consensus metadata
I20260809 06:28:26.954613 28463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:27.073896 28168 catalog_manager.cc:4841] TS 2d638e4c7e874680ba2594305c2175a9 (127.8.131.1:38741): tablet 02d5e7ac5285463f853677690a2fd801 (table test-table [id=ec145db8fa904ec588ea2185f33605da]) successfully deleted
I20260809 06:28:27.073961 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 28247
I20260809 06:28:27.098181 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 28474
I20260809 06:28:27.112587 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:27.099175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59326 (local address 127.8.131.2:39113)
0809 06:28:27.099386 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:27.099394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:27.099433 (+ 39us) server_negotiation.cc:407] Connection header received
0809 06:28:27.099512 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:27.099516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:27.099580 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:27.099719 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:27.101055 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:27.101887 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:27.103198 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:27.103696 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:27.103838 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:27.106365 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:27.106377 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:27.106394 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:27.106436 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:27.109528 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:27.110335 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:27.110342 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:27.110344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:27.110530 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:27.110916 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:27.110923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:27.110927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:27.111308 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:27.111642 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:27.111975 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 06:28:27.112224 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 06:28:27.113313 28666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:27.236855 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 28474
I20260809 06:28:27.265745 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 28681
I20260809 06:28:27.278735 28884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:27.266593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37330 (local address 127.8.131.3:44225)
0809 06:28:27.266759 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:27.266764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:27.266788 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:28:27.266877 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:27.266880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:27.266958 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:27.267046 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:27.268565 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:27.269286 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:27.270471 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:27.270843 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:27.270933 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:27.273393 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:27.273405 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:27.273419 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:27.273455 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:27.276442 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:27.277046 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:27.277053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:27.277055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:27.277183 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:27.277487 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:27.277490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:27.277492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:27.277830 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:27.278057 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:27.278297 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 06:28:27.278454 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 06:28:27.279433 28852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:27.367766 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 28681
I20260809 06:28:27.391021 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 28147
I20260809 06:28:27.392222 28235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:27.504583 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 28147
I20260809 06:28:27.537474 8716 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 06:28:27.883453 8716 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 06:28:27.883773 8716 mini_ranger.cc:66] Stopping Ranger...
I20260809 06:28:28.229775 8716 mini_ranger.cc:68] Stopped Ranger
2026-08-09 06:28:28.230 UTC [26843] LOG: received smart shutdown request
2026-08-09 06:28:28.233 UTC [26843] LOG: background worker "logical replication launcher" (PID 26853) exited with exit code 1
2026-08-09 06:28:28.233 UTC [26847] LOG: shutting down
2026-08-09 06:28:28.376 UTC [26843] LOG: database system is shut down
2026-08-09T06:28:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54463 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:28:28 dist-test-slave-mjjr krb5kdc[28949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:28:28 dist-test-slave-mjjr krb5kdc[28949](info): set up 2 sockets
Aug 09 06:28:28 dist-test-slave-mjjr krb5kdc[28949](info): commencing operation
krb5kdc: starting...
W20260809 06:28:29.802587 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:28:29 dist-test-slave-mjjr krb5kdc[28949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:28:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:28:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:28:30.073237 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41307
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:41307,127.8.131.61:38385,127.8.131.60:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:30.414796 28965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:30.415108 28965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:30.415215 28965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:30.422639 28965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:28:30.422762 28965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:30.422816 28965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:30.422868 28965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:28:30.422904 28965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:28:30.432897 28965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:41307,127.8.131.61:38385,127.8.131.60:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:30.434190 28965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:30.435451 28965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:30.447772 28970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:30.447811 28971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:30.447850 28974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:30.448721 28965 server_base.cc:1034] running on GCE node
I20260809 06:28:30.449512 28965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:30.450904 28965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:30.452140 28965 hybrid_clock.cc:648] HybridClock initialized: now 1786256910452102 us; error 36 us; skew 500 ppm
I20260809 06:28:30.452486 28965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:30 dist-test-slave-mjjr krb5kdc[28949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256910, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:30.459489 28965 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:28:30.461128 28965 webserver.cc:460] Webserver started at http://127.8.131.62:36081/ using document root <none> and password file <none>
I20260809 06:28:30.461823 28965 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:30.461985 28965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:30.462289 28965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:28:30.465322 28965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "1a6571eae96b4698b025e247ebf671d3"
format_stamp: "Formatted at 2026-08-09 06:28:30 on dist-test-slave-mjjr"
server_key: "083f4e899c46bece9210998107fa57a6"
server_key_iv: "ef3b926070f7c2a7b9cfc334cdd01de6"
server_key_version: "encryptionkey@0"
I20260809 06:28:30.466181 28965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "1a6571eae96b4698b025e247ebf671d3"
format_stamp: "Formatted at 2026-08-09 06:28:30 on dist-test-slave-mjjr"
server_key: "083f4e899c46bece9210998107fa57a6"
server_key_iv: "ef3b926070f7c2a7b9cfc334cdd01de6"
server_key_version: "encryptionkey@0"
I20260809 06:28:30.471256 28965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:28:30.474627 28987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:30.476033 28965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:28:30.476218 28965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "1a6571eae96b4698b025e247ebf671d3"
format_stamp: "Formatted at 2026-08-09 06:28:30 on dist-test-slave-mjjr"
server_key: "083f4e899c46bece9210998107fa57a6"
server_key_iv: "ef3b926070f7c2a7b9cfc334cdd01de6"
server_key_version: "encryptionkey@0"
I20260809 06:28:30.476416 28965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:30.513314 28965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:30.517329 28965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:30.517639 28965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:30.536684 28965 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:41307
I20260809 06:28:30.536725 29051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:41307 every 8 connection(s)
I20260809 06:28:30.538069 28965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:28:30.540274 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 28965
I20260809 06:28:30.540542 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:28:30.541404 8716 external_mini_cluster.cc:1428] Setting key 221564a3b66c94e4b83ab3ab2dd07d8c
I20260809 06:28:30.542514 29052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:30.552620 29052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:28:30 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256910, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.131.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.61@KRBTEST.COM" created.
I20260809 06:28:30.575042 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.555814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41307 (local address 127.0.0.1:42400)
0809 06:28:30.556999 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.557032 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:30.558767 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:30.559587 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:30.559594 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:30.560332 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:30.560661 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:30.560677 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:30.562219 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:30.562224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:30.563273 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:30.563285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:30.564283 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:30.564290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:30.564537 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:30.565198 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:30.565234 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:30.567633 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:30.571205 (+ 3572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:30.571215 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:30.571225 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:30.571602 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:30.572129 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:30.572132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:30.572134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:30.572275 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:30.573236 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:30.573241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:30.573421 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 06:28:30.573618 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 06:28:30.573991 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"mutex_wait_us":47,"spinlock_wait_cycles":16256,"thread_start_us":752,"threads_started":1}
I20260809 06:28:30.575088 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.555813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42400 (local address 127.8.131.62:41307)
0809 06:28:30.556682 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:30.556699 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:30.557174 (+ 475us) server_negotiation.cc:407] Connection header received
0809 06:28:30.559131 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:30.559136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:30.559345 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:30.559555 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:30.560862 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:30.562063 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:30.563442 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:30.564042 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:30.564225 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:30.567932 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:30.567940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:30.567952 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:30.567990 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:30.571067 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:30.571805 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:30.571817 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:30.571824 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:30.572020 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:30.572392 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:30.572399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:30.572405 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:30.573100 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:30.573311 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:30.573810 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 06:28:30.574143 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":626,"thread_start_us":407,"threads_started":1}
I20260809 06:28:30.578471 29052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:28:30.579753 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.579231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47166)
0809 06:28:30.579418 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.579562 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 06:28:30.580065 28988 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
I20260809 06:28:30.581831 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.581403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47176)
0809 06:28:30.581582 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.581674 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 06:28:30.582170 29052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:38385: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab.
Entry for principal kudu/127.8.131.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab.
I20260809 06:28:30.601164 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.61:38385
--webserver_interface=127.8.131.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab
--principal=kudu/127.8.131.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:41307,127.8.131.61:38385,127.8.131.60:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:28:30.631613 29052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } attempt: 1
I20260809 06:28:30.632982 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.632361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47190)
0809 06:28:30.632620 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.632779 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 06:28:30.634080 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.633662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47196)
0809 06:28:30.633825 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.633920 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 06:28:30.634277 29052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:38385: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
I20260809 06:28:30.702553 29052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } attempt: 2
I20260809 06:28:30.704015 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.703335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47210)
0809 06:28:30.703632 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.703811 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 06:28:30.705260 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.704725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47216)
0809 06:28:30.704930 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.705042 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 06:28:30.705506 29052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:38385: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
I20260809 06:28:30.796798 29052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } attempt: 3
I20260809 06:28:30.798441 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.797786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47220)
0809 06:28:30.798069 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.798212 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 06:28:30.799829 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.799251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47222)
0809 06:28:30.799439 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.799565 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 06:28:30.800071 29052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:38385: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
W20260809 06:28:30.942992 29064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:30.943362 29064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260809 06:28:30.943367 29052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } attempt: 4
W20260809 06:28:30.943514 29064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 06:28:30.944679 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.944037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47226)
0809 06:28:30.944380 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.944498 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260809 06:28:30.945727 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:30.945319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47230)
0809 06:28:30.945473 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:30.945569 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 06:28:30.945925 29052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:38385: Network error: Client connection negotiation failed: client connection to 127.8.131.61:38385: connect: Connection refused (error 111)
W20260809 06:28:30.951831 29064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:28:30.951932 29064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:30.951983 29064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:30.952041 29064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:28:30.952080 29064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:28:30.961926 29064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:41307,127.8.131.61:38385,127.8.131.60:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab
--principal=kudu/127.8.131.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.61:38385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.131.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:30.963163 29064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:30.964491 29064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:30.976277 29072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:30.976315 29073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:30.976361 29075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:30.980835 29064 server_base.cc:1034] running on GCE node
I20260809 06:28:30.981702 29064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:30.983032 29064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:30.984258 29064 hybrid_clock.cc:648] HybridClock initialized: now 1786256910984221 us; error 54 us; skew 500 ppm
I20260809 06:28:30.984648 29064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:30 dist-test-slave-mjjr krb5kdc[28949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256910, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:30.987861 29064 init.cc:377] Logged in from keytab as kudu/127.8.131.61@KRBTEST.COM (short username kudu)
I20260809 06:28:30.989328 29064 webserver.cc:460] Webserver started at http://127.8.131.61:35531/ using document root <none> and password file <none>
I20260809 06:28:30.990080 29064 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:30.990229 29064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:30.990535 29064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:28:30.993552 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/data/instance:
uuid: "7222123c670449efa1837d3fe5934cf5"
format_stamp: "Formatted at 2026-08-09 06:28:30 on dist-test-slave-mjjr"
server_key: "aff58bec3405231a07c7f1634b78b13a"
server_key_iv: "224beb234744b088b99e92ba03fe2255"
server_key_version: "encryptionkey@0"
I20260809 06:28:30.994433 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/wal/instance:
uuid: "7222123c670449efa1837d3fe5934cf5"
format_stamp: "Formatted at 2026-08-09 06:28:30 on dist-test-slave-mjjr"
server_key: "aff58bec3405231a07c7f1634b78b13a"
server_key_iv: "224beb234744b088b99e92ba03fe2255"
server_key_version: "encryptionkey@0"
I20260809 06:28:30.999703 29064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 06:28:31.003100 29085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:31.004499 29064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 06:28:31.004685 29064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/wal
uuid: "7222123c670449efa1837d3fe5934cf5"
format_stamp: "Formatted at 2026-08-09 06:28:30 on dist-test-slave-mjjr"
server_key: "aff58bec3405231a07c7f1634b78b13a"
server_key_iv: "224beb234744b088b99e92ba03fe2255"
server_key_version: "encryptionkey@0"
I20260809 06:28:31.004892 29064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:31.035918 29064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:31.039924 29064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:31.040220 29064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:31.058800 29064 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.61:38385
I20260809 06:28:31.058831 29158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.61:38385 every 8 connection(s)
I20260809 06:28:31.060240 29064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
I20260809 06:28:31.063966 29159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:31.066998 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 29064
I20260809 06:28:31.067194 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-1/wal/instance
I20260809 06:28:31.067709 8716 external_mini_cluster.cc:1428] Setting key 85dfa1c61e2f09302deddb4961529b10
I20260809 06:28:31.070434 29159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256910, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:31.090354 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.073023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.8.131.62:41307)
0809 06:28:31.073199 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.073203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.074571 (+ 1368us) server_negotiation.cc:407] Connection header received
0809 06:28:31.076474 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.076480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.076574 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.076689 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.078151 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.078837 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.080510 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.081008 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.081142 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.084582 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.084587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.084590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.084622 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.087317 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.088128 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.088132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.088135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.088249 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.088677 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.088681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.088689 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.088968 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.089135 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.089805 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.090048 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.8.131.60@KRBTEST.COM; defaulting to no policy
I20260809 06:28:31.091332 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.073611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41307 (local address 127.0.0.1:42410)
0809 06:28:31.074398 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.074441 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:31.076191 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:31.076928 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:31.076935 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:31.077435 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:31.077975 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.077991 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.079009 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.079013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.080362 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.080374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.081239 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.081246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.081493 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.082163 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:31.082185 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:31.084355 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:31.087470 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.087479 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.087494 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.087999 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.088356 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.088361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.088365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.088557 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.089107 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:31.089115 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:31.089446 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 06:28:31.089697 (+ 251us) client_negotiation.cc:240] Negotiation successful
0809 06:28:31.090173 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":370,"threads_started":1}
I20260809 06:28:31.093032 29159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.60@KRBTEST.COM" created.
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256910, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:28:31.107266 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.093850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47234)
0809 06:28:31.094118 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.094133 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:31.094213 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:31.095101 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:31.095105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:31.095531 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:31.095705 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.095716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.097175 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.097183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.098009 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.098020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.098807 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.098813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.099014 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.099548 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:31.099564 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:31.101774 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:31.104858 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.104863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.104867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.105200 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.105569 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.105576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.105581 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.105713 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.106278 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:31.106283 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:31.106453 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 06:28:31.106610 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 06:28:31.106881 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260809 06:28:31.107403 29165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.094058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.8.131.61:38385)
0809 06:28:31.094735 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.094741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.094763 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:28:31.094802 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.094807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.094958 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.095110 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.095859 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.097051 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.098143 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.098665 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.098790 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.101987 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.101997 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.102008 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.102035 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.104724 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.105325 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.105337 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.105339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.105461 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.105814 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.105820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.105823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.106160 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.106372 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.106740 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.107006 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":547,"thread_start_us":406,"threads_started":1}
I20260809 06:28:31.109936 29159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:28:31.111178 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.110601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:40966)
0809 06:28:31.110824 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.110963 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 06:28:31.111438 29088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
I20260809 06:28:31.113099 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.112679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:40982)
0809 06:28:31.112857 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.112952 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 06:28:31.113318 29159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:43305: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab.
Entry for principal kudu/127.8.131.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab.
I20260809 06:28:31.124950 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.60:43305
--webserver_interface=127.8.131.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab
--principal=kudu/127.8.131.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:41307,127.8.131.61:38385,127.8.131.60:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:28:31.162655 29159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } attempt: 1
I20260809 06:28:31.163977 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.163466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:40984)
0809 06:28:31.163713 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.163817 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 06:28:31.164950 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.164648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:40986)
0809 06:28:31.164767 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.164841 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 06:28:31.165222 29159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:43305: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
I20260809 06:28:31.233547 29159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } attempt: 2
I20260809 06:28:31.234863 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.234419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:40996)
0809 06:28:31.234613 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.234706 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 06:28:31.236080 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.235712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41006)
0809 06:28:31.235886 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.235959 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 06:28:31.236279 29159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:43305: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
I20260809 06:28:31.245155 29052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } attempt: 5
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256910, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:28:31.260134 29165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.245919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.8.131.61:38385)
0809 06:28:31.246079 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.246085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.246171 (+ 86us) server_negotiation.cc:407] Connection header received
0809 06:28:31.246399 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.246404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.246491 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.246598 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.247414 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.248167 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.249579 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.250072 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.250215 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.253477 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.253482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.253486 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.253517 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.257034 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.257928 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.257934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.257937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.258108 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.258625 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.258630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.258633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.258985 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.259150 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.259555 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.259809 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:28:31.260144 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.245854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47242)
0809 06:28:31.246055 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.246077 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:31.246254 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:31.246632 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:31.246636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:31.247066 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:31.247256 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.247265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.248499 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.248505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.249426 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.249439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.250242 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.250248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.250429 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.251066 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:31.251086 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:31.253279 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:31.257217 (+ 3938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.257222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.257226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.257798 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.258299 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.258304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.258308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.258485 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.259116 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:31.259121 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:31.259290 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 06:28:31.259444 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 06:28:31.259795 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 06:28:31.261477 29052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:28:31.262509 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.262131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41020)
0809 06:28:31.262290 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.262378 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 06:28:31.263398 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.263061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41034)
0809 06:28:31.263219 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.263298 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 06:28:31.263592 29052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:43305: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
I20260809 06:28:31.314872 29052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } attempt: 1
I20260809 06:28:31.316064 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.315564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41044)
0809 06:28:31.315786 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.315890 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 06:28:31.317070 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.316715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41048)
0809 06:28:31.316891 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.316969 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 06:28:31.317289 29052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:43305: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
I20260809 06:28:31.327507 29159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } attempt: 3
I20260809 06:28:31.328743 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.328304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41058)
0809 06:28:31.328511 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.328597 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 06:28:31.329778 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.329352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41066)
0809 06:28:31.329599 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.329671 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260809 06:28:31.329960 29159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:43305: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
I20260809 06:28:31.385485 29052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } attempt: 2
I20260809 06:28:31.386655 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.386126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41082)
0809 06:28:31.386346 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.386447 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 06:28:31.387779 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.387406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41084)
0809 06:28:31.387563 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.387628 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 06:28:31.388017 29052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:43305: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
I20260809 06:28:31.473157 29159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } attempt: 4
I20260809 06:28:31.474387 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.473925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41090)
0809 06:28:31.474147 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.474237 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 06:28:31.474363 29172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:31.474792 29172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:31.474967 29172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 06:28:31.475463 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.475054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41094)
0809 06:28:31.475228 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.475322 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 06:28:31.475757 29159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:43305: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
W20260809 06:28:31.482678 29172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:28:31.482816 29172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:31.482915 29172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:31.483050 29172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:28:31.483136 29172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:28:31.493443 29172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:41307,127.8.131.61:38385,127.8.131.60:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab
--principal=kudu/127.8.131.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.60:43305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.131.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:31.494233 29052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } attempt: 3
I20260809 06:28:31.494941 29172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:31.495461 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.494966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41102)
0809 06:28:31.495229 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.495314 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 06:28:31.496488 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.496152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41106)
0809 06:28:31.496324 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.496399 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 06:28:31.496497 29172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:31.496727 29052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:43305: Network error: Client connection negotiation failed: client connection to 127.8.131.60:43305: connect: Connection refused (error 111)
W20260809 06:28:31.508669 29179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:31.508716 29180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:31.508982 29183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:31.509935 29172 server_base.cc:1034] running on GCE node
I20260809 06:28:31.510747 29172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:31.512172 29172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:31.513435 29172 hybrid_clock.cc:648] HybridClock initialized: now 1786256911513377 us; error 49 us; skew 500 ppm
I20260809 06:28:31.513850 29172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256911, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:31.516965 29172 init.cc:377] Logged in from keytab as kudu/127.8.131.60@KRBTEST.COM (short username kudu)
I20260809 06:28:31.518342 29172 webserver.cc:460] Webserver started at http://127.8.131.60:45913/ using document root <none> and password file <none>
I20260809 06:28:31.519017 29172 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:31.519163 29172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:31.519471 29172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:28:31.522481 29172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/data/instance:
uuid: "bda80f9b95ac41499051b563bfa33ff8"
format_stamp: "Formatted at 2026-08-09 06:28:31 on dist-test-slave-mjjr"
server_key: "f840b171378351bb576a9665d2fafa77"
server_key_iv: "2ad89f55ec0435a7ad46375808037c27"
server_key_version: "encryptionkey@0"
I20260809 06:28:31.523303 29172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/wal/instance:
uuid: "bda80f9b95ac41499051b563bfa33ff8"
format_stamp: "Formatted at 2026-08-09 06:28:31 on dist-test-slave-mjjr"
server_key: "f840b171378351bb576a9665d2fafa77"
server_key_iv: "2ad89f55ec0435a7ad46375808037c27"
server_key_version: "encryptionkey@0"
I20260809 06:28:31.528352 29172 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 06:28:31.531615 29193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:31.532958 29172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:28:31.533241 29172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/wal
uuid: "bda80f9b95ac41499051b563bfa33ff8"
format_stamp: "Formatted at 2026-08-09 06:28:31 on dist-test-slave-mjjr"
server_key: "f840b171378351bb576a9665d2fafa77"
server_key_iv: "2ad89f55ec0435a7ad46375808037c27"
server_key_version: "encryptionkey@0"
I20260809 06:28:31.533442 29172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:31.551802 29172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:31.555759 29172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:31.556079 29172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:31.574712 29172 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.60:43305
I20260809 06:28:31.574754 29265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.60:43305 every 8 connection(s)
I20260809 06:28:31.576131 29172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
I20260809 06:28:31.579844 29266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:31.580097 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 29172
I20260809 06:28:31.580291 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786256699676692-8716-0/minicluster-data/master-2/wal/instance
I20260809 06:28:31.580925 8716 external_mini_cluster.cc:1428] Setting key d26a9b5b1da97b917d40bc4ff8d0d05d
I20260809 06:28:31.589011 29266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256911, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:31.604961 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.582825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.8.131.60:43305)
0809 06:28:31.583732 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.583744 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.583784 (+ 40us) server_negotiation.cc:407] Connection header received
0809 06:28:31.585925 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.585946 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.586313 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.586675 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.588452 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.589699 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.591811 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.592815 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.593036 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.597122 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.597130 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.597140 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.597179 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.600551 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.601271 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.601277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.601284 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.601435 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.601841 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.601846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.601848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.602353 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.602662 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.603116 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.603540 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":425,"threads_started":1}
I20260809 06:28:31.606184 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.591762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.8.131.62:41307)
0809 06:28:31.592298 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.592304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.593141 (+ 837us) server_negotiation.cc:407] Connection header received
0809 06:28:31.593200 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.593204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.593264 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.593380 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.594459 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.595168 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.596589 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.597002 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.597146 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.600662 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.600667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.600670 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.600703 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.603319 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.604243 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.604247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.604249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.604385 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.604808 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.604811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.604813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.605049 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.605204 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.605728 (+ 524us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.605896 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":266,"threads_started":1}
I20260809 06:28:31.606356 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.591731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:41307 (local address 127.0.0.1:42422)
0809 06:28:31.592823 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.592874 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:31.593039 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:31.593444 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:31.593451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:31.594040 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:31.594291 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.594299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.595324 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.595335 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.596453 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.596465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.597198 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.597204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.597455 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.598217 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:31.598234 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:31.600469 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:31.603525 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.603535 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.603541 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.604112 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.604511 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.604515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.604518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.604698 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.605225 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:31.605233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:31.605426 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 06:28:31.605631 (+ 205us) client_negotiation.cc:240] Negotiation successful
0809 06:28:31.605920 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"thread_start_us":674,"threads_started":1}
I20260809 06:28:31.607867 29266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256911, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:31.621294 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.608625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:38385 (local address 127.0.0.1:47256)
0809 06:28:31.608807 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.608827 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:31.608914 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:31.609228 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:31.609232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:31.609630 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:31.609766 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.609774 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.610789 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.610795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.611725 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.611736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.612445 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.612451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.612656 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.613227 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:31.613247 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:31.615542 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:31.618935 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.618940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.618944 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.619330 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.619734 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.619739 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.619742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.619890 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.620385 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:31.620390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:31.620537 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 06:28:31.620674 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 06:28:31.620950 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 06:28:31.621333 29165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.608699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.8.131.61:38385)
0809 06:28:31.608922 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.608928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.608949 (+ 21us) server_negotiation.cc:407] Connection header received
0809 06:28:31.609013 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.609018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.609101 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.609207 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.609916 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.610656 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.611864 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.612285 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.612411 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.615788 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.615798 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.615803 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.615862 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.618798 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.619448 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.619454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.619458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.619589 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.620004 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.620010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.620014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.620269 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.620429 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.620766 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.620980 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 06:28:31.622532 29266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:28:31.622714 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.608566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.8.131.62:41307)
0809 06:28:31.608878 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.608882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.608896 (+ 14us) server_negotiation.cc:407] Connection header received
0809 06:28:31.608936 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.608939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.609008 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.609120 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.610420 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.611081 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.612384 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.612787 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.612921 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.617279 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.617284 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.617287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.617317 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.619877 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.620521 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.620527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.620530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.620653 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.621041 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.621046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.621049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.621314 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.621618 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.622156 (+ 538us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.622370 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256911, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:28:31.635864 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.623340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41130)
0809 06:28:31.623977 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.623996 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:31.624103 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:31.624405 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:31.624410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:31.624879 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:31.625015 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.625023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.625939 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.625944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.626800 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.626810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.627534 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.627539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.627770 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.628318 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:31.628334 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:31.630214 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:31.633317 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.633323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.633327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.633762 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.634133 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.634138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.634143 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.634309 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.634824 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:31.634829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:31.634980 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 06:28:31.635121 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 06:28:31.635418 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515}
I20260809 06:28:31.635917 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.623482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.8.131.60:43305)
0809 06:28:31.623684 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.623689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.624075 (+ 386us) server_negotiation.cc:407] Connection header received
0809 06:28:31.624183 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.624188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.624256 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.624357 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.625149 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.625812 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.626977 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.627374 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.627520 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.630374 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.630380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.630385 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.630427 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.633195 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.633880 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.633899 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.633901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.634021 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.634436 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.634441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.634444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.634709 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.634861 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.635210 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.635571 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":62}
I20260809 06:28:31.639817 29165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.624729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47272 (local address 127.8.131.61:38385)
0809 06:28:31.624879 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.624896 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.624914 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:28:31.625093 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.625098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.625164 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.625261 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.626474 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.627169 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.628658 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.629107 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.629243 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.633695 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.633703 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.633708 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.633745 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.637084 (+ 3339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.637835 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.637841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.637844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.637995 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.638409 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.638414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.638417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.638680 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.638941 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.639240 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.639479 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 06:28:31.641016 29266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8: Bootstrap starting.
I20260809 06:28:31.644682 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:28:31.644730 29266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:31.645715 29266 log.cc:824] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:31.648623 29266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8: No bootstrap required, opened a new log
I20260809 06:28:31.662575 29266 raft_consensus.cc:348] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } }
I20260809 06:28:31.663336 29266 raft_consensus.cc:374] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:31.663547 29266 raft_consensus.cc:729] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda80f9b95ac41499051b563bfa33ff8, State: Initialized, Role: FOLLOWER
I20260809 06:28:31.664384 29266 consensus_queue.cc:260] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } }
I20260809 06:28:31.675297 29052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } attempt: 4
I20260809 06:28:31.677876 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.650479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42436 (local address 127.8.131.62:41307)
0809 06:28:31.650664 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.650670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.657740 (+ 7070us) server_negotiation.cc:407] Connection header received
0809 06:28:31.658218 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.658224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.658313 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.658385 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.661137 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.662360 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.665354 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.666035 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.666143 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.670564 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.670571 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.670576 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.670622 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.674276 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.675267 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.675278 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.675283 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.675448 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.675972 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.675978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.675981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.676401 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.676660 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.677208 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.677453 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 06:28:31.678954 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.650034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.8.131.60:43305)
0809 06:28:31.650212 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.650219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.660335 (+ 10116us) server_negotiation.cc:407] Connection header received
0809 06:28:31.660516 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.660524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.660633 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.660731 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.662913 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.664054 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.666346 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.667152 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.667265 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.671086 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.671093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.671098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.671140 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.675141 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.676365 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.676370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.676374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.676529 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.677020 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.677028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.677030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.677453 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.677722 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.678301 (+ 579us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.678527 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256910, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
I20260809 06:28:31.686899 29284 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:31.688647 29266 sys_catalog.cc:564] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:28:31.690231 29165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.649954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47286 (local address 127.8.131.61:38385)
0809 06:28:31.650376 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.650383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.662533 (+ 12150us) server_negotiation.cc:407] Connection header received
0809 06:28:31.662604 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.662610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.662692 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.662813 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.665298 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.666775 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.669845 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.670752 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.670878 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.683323 (+ 12445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.683332 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.683340 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.683398 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.687093 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.688020 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.688026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.688029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.688193 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.688685 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.688690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.688694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.689020 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.689262 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.689610 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.689853 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290}
I20260809 06:28:31.688359 29284 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:28:31.695153 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.676796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41152 (local address 127.8.131.60:43305)
0809 06:28:31.679377 (+ 2581us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.679384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.679403 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:28:31.679449 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.679453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.679545 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.680573 (+ 1028us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.680690 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.681521 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.682873 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.683545 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.684241 (+ 696us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.687740 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.687747 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.687753 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.687793 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.691811 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.692787 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.692795 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.692801 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.692999 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.693462 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.693469 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.693474 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.693945 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.694122 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.694755 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.694916 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2479,"thread_start_us":16407,"threads_started":1}
I20260809 06:28:31.695577 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.675946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41152)
0809 06:28:31.676392 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.676429 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:31.676556 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:31.679744 (+ 3188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:31.679750 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:31.680230 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:31.680426 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.680438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.681648 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.681653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.682720 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.682745 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.683748 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.683756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.683957 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.684666 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:31.684690 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:31.687496 (+ 2806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:31.691963 (+ 4467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.691972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.691978 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.692640 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.693142 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.693149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.693153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.693333 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.694218 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:31.694223 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:31.694441 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 06:28:31.694645 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 06:28:31.695015 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285}
I20260809 06:28:31.701221 29052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3: Bootstrap starting.
I20260809 06:28:31.705143 29052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:31.706488 29052 log.cc:824] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3: Log is configured to *not* fsync() on all Append() calls
W20260809 06:28:31.706496 29303 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:28:31.706626 29303 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:28:31.710458 29052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3: No bootstrap required, opened a new log
I20260809 06:28:31.702687 29172 master_runner.cc:186] Error getting master registration for 127.8.131.62:41307: OK, instance_id { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" instance_seqno: 1786256910531348 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 06:28:31.714409 29172 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 06:28:31.719554 29052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } }
I20260809 06:28:31.719928 29052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:31.720069 29052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a6571eae96b4698b025e247ebf671d3, State: Initialized, Role: FOLLOWER
I20260809 06:28:31.720703 29052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } }
I20260809 06:28:31.721671 29307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:31.722101 29307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:28:31.722810 29052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 06:28:31.733775 29322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:28:31.733999 29322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:28:31.736420 28965 master_runner.cc:186] Error getting master registration for 127.8.131.61:38385: OK, instance_id { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" instance_seqno: 1786256911053514 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 06:28:31.749475 28965 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 06:28:31.770576 28997 tablet.cc:2369] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:28:31.775025 29159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } attempt: 5
Aug 09 06:28:31 dist-test-slave-mjjr krb5kdc[28949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256910, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
I20260809 06:28:31.794626 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.775694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:43305 (local address 127.0.0.1:41156)
0809 06:28:31.775976 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:31.775995 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:31.776125 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:31.776542 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:31.776547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:31.777124 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:31.777340 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.777352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.778854 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.778860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.780359 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:31.780375 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.782872 (+ 2497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.782879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:31.783132 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.783948 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:31.783990 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:31.787008 (+ 3018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:31.791405 (+ 4397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.791411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.791415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.791988 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.792487 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:31.792493 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:31.792497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:31.792677 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:31.793419 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:31.793425 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:31.793602 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 06:28:31.793770 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 06:28:31.794097 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260809 06:28:31.794631 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:31.775779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.8.131.60:43305)
0809 06:28:31.775991 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:31.775998 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:31.776053 (+ 55us) server_negotiation.cc:407] Connection header received
0809 06:28:31.776287 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:31.776294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:31.776379 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:31.776495 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:31.777500 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.778697 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.781700 (+ 3003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:31.782658 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:31.782855 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:31.787221 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:31.787228 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:31.787234 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:31.787279 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:31.791254 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.792138 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.792144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.792148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.792320 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:31.792828 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:31.792833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:31.792836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:31.793263 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:31.793453 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:31.793926 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 06:28:31.794182 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 06:28:31.801820 29159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5: Bootstrap starting.
I20260809 06:28:31.805927 29159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:31.807261 29159 log.cc:824] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5: Log is configured to *not* fsync() on all Append() calls
W20260809 06:28:31.808115 29202 tablet.cc:2369] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:28:31.811902 29159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5: No bootstrap required, opened a new log
I20260809 06:28:31.823120 29159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } }
I20260809 06:28:31.823575 29159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:31.823738 29159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7222123c670449efa1837d3fe5934cf5, State: Initialized, Role: FOLLOWER
I20260809 06:28:31.824478 29159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } }
I20260809 06:28:31.825518 29325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:31.825999 29325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:28:31.827073 29159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:28:31.853070 29064 master_runner.cc:263] No leader master found from master 7222123c670449efa1837d3fe5934cf5
I20260809 06:28:31.853237 29064 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 06:28:31.858407 29336 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:28:31.858644 29336 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:28:31.883102 29307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:28:31.883460 29307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } }
I20260809 06:28:31.885155 29307 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7222123c670449efa1837d3fe5934cf5 (127.8.131.61:38385), bda80f9b95ac41499051b563bfa33ff8 (127.8.131.60:43305)
I20260809 06:28:31.886322 29122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6571eae96b4698b025e247ebf671d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7222123c670449efa1837d3fe5934cf5" is_pre_election: true
I20260809 06:28:31.886987 29122 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a6571eae96b4698b025e247ebf671d3 in term 0.
I20260809 06:28:31.888024 28988 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a6571eae96b4698b025e247ebf671d3, 7222123c670449efa1837d3fe5934cf5; no voters:
I20260809 06:28:31.889078 29229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6571eae96b4698b025e247ebf671d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda80f9b95ac41499051b563bfa33ff8" is_pre_election: true
I20260809 06:28:31.889691 29229 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a6571eae96b4698b025e247ebf671d3 in term 0.
I20260809 06:28:31.890671 29307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:28:31.890823 29307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:28:31.890926 29307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:31.894382 29307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } }
I20260809 06:28:31.896124 29122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6571eae96b4698b025e247ebf671d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7222123c670449efa1837d3fe5934cf5"
I20260809 06:28:31.896394 29122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:31.897161 29229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a6571eae96b4698b025e247ebf671d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bda80f9b95ac41499051b563bfa33ff8"
I20260809 06:28:31.897408 29229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:31.899334 29307 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [CANDIDATE]: Term 1 election: Requested vote from peers 7222123c670449efa1837d3fe5934cf5 (127.8.131.61:38385), bda80f9b95ac41499051b563bfa33ff8 (127.8.131.60:43305)
I20260809 06:28:31.900991 29229 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a6571eae96b4698b025e247ebf671d3 in term 1.
I20260809 06:28:31.902926 29122 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a6571eae96b4698b025e247ebf671d3 in term 1.
I20260809 06:28:31.902966 28991 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a6571eae96b4698b025e247ebf671d3, bda80f9b95ac41499051b563bfa33ff8; no voters:
I20260809 06:28:31.903376 29307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:28:31.912241 29307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [term 1 LEADER]: Becoming Leader. State: Replica: 1a6571eae96b4698b025e247ebf671d3, State: Running, Role: LEADER
I20260809 06:28:31.912894 29307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } }
I20260809 06:28:31.919919 29339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6571eae96b4698b025e247ebf671d3. Latest consensus state: current_term: 1 leader_uuid: "1a6571eae96b4698b025e247ebf671d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:31.920377 29339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:31.930354 29343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:28:31.932619 29343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:28:31.952239 29229 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [term 1 FOLLOWER]: Refusing update from remote peer 1a6571eae96b4698b025e247ebf671d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:28:31.953300 29339 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:28:31.960513 29122 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [term 1 FOLLOWER]: Refusing update from remote peer 1a6571eae96b4698b025e247ebf671d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:28:31.961360 29339 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:28:31.993652 29325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6571eae96b4698b025e247ebf671d3. Latest consensus state: current_term: 1 leader_uuid: "1a6571eae96b4698b025e247ebf671d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:31.994022 29325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:28:31.998157 29344 mvcc.cc:205] Tried to move back new op lower bound from 7316508311329112064 to 7316508311200231424. Current Snapshot: MvccSnapshot[applied={T|T < 7316508311329112064}]
I20260809 06:28:32.000080 29339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a6571eae96b4698b025e247ebf671d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:32.000360 29339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:32.004464 29347 mvcc.cc:205] Tried to move back new op lower bound from 7316508311329112064 to 7316508311200231424. Current Snapshot: MvccSnapshot[applied={T|T < 7316508311329112064}]
I20260809 06:28:32.004691 29284 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6571eae96b4698b025e247ebf671d3. Latest consensus state: current_term: 1 leader_uuid: "1a6571eae96b4698b025e247ebf671d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:32.005009 29284 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:28:32.007330 29307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a6571eae96b4698b025e247ebf671d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:32.007673 29307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3 [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:32.008200 29325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a6571eae96b4698b025e247ebf671d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:32.008447 29325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:28:32.012389 29343 catalog_manager.cc:1347] Generated new cluster ID: 3d9d5704705942fd97c922d0121c8a2a
I20260809 06:28:32.012501 29343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:28:32.031422 29345 mvcc.cc:205] Tried to move back new op lower bound from 7316508311329112064 to 7316508311200231424. Current Snapshot: MvccSnapshot[applied={T|T < 7316508311329112064}]
I20260809 06:28:32.037205 29284 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a6571eae96b4698b025e247ebf671d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a6571eae96b4698b025e247ebf671d3" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41307 } } peers { permanent_uuid: "7222123c670449efa1837d3fe5934cf5" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 38385 } } peers { permanent_uuid: "bda80f9b95ac41499051b563bfa33ff8" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 43305 } } }
I20260809 06:28:32.037501 29284 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:28:32.067276 29343 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:28:32.069285 29343 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:28:32.109995 29343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a6571eae96b4698b025e247ebf671d3: Generated new TSK 0
I20260809 06:28:32.111235 29343 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:28:32.708748 29303 catalog_manager.cc:1259] Loaded cluster ID: 3d9d5704705942fd97c922d0121c8a2a
I20260809 06:28:32.708846 29303 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8: loading cluster ID for follower catalog manager: success
I20260809 06:28:32.711321 29303 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8: acquiring CA information for follower catalog manager: success
I20260809 06:28:32.713050 29303 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bda80f9b95ac41499051b563bfa33ff8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:28:32.860719 29336 catalog_manager.cc:1259] Loaded cluster ID: 3d9d5704705942fd97c922d0121c8a2a
I20260809 06:28:32.860829 29336 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5: loading cluster ID for follower catalog manager: success
I20260809 06:28:32.863263 29336 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5: acquiring CA information for follower catalog manager: success
I20260809 06:28:32.865249 29336 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7222123c670449efa1837d3fe5934cf5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:28:33.251957 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 28965
I20260809 06:28:33.253317 29046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:33.367074 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 28965
I20260809 06:28:33.396672 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 29064
I20260809 06:28:33.397935 29152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:33.505460 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 29064
I20260809 06:28:33.533841 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 29172
I20260809 06:28:33.535079 29256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:33.643437 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 29172
2026-08-09T06:28:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5140 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:28:33 dist-test-slave-mjjr krb5kdc[29371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:28:33 dist-test-slave-mjjr krb5kdc[29371](info): set up 2 sockets
Aug 09 06:28:33 dist-test-slave-mjjr krb5kdc[29371](info): commencing operation
krb5kdc: starting...
W20260809 06:28:34.977931 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:28:35 dist-test-slave-mjjr krb5kdc[29371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:28:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:28:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:28:35.251240 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45795
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:45795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:35.594182 29391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:35.594480 29391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:35.594573 29391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:35.601951 29391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:28:35.602073 29391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:35.602123 29391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:35.602183 29391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:28:35.602229 29391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:28:35.612284 29391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:45795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:45795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:35.613579 29391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:35.614909 29391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:35.627964 29400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:35.628005 29398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:35.628068 29397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:35.628925 29391 server_base.cc:1034] running on GCE node
I20260809 06:28:35.629652 29391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:35.631049 29391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:35.632284 29391 hybrid_clock.cc:648] HybridClock initialized: now 1786256915632243 us; error 52 us; skew 500 ppm
I20260809 06:28:35.632647 29391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:35 dist-test-slave-mjjr krb5kdc[29371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256915, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:35.638537 29391 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:28:35.640223 29391 webserver.cc:460] Webserver started at http://127.8.131.62:44705/ using document root <none> and password file <none>
I20260809 06:28:35.640956 29391 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:35.641119 29391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:35.641448 29391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:28:35.644613 29391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "2229dfb877084b319ef5305479375118"
format_stamp: "Formatted at 2026-08-09 06:28:35 on dist-test-slave-mjjr"
server_key: "b5b51a73707aef87aeba4e731905ea8b"
server_key_iv: "904f564350b88a908442ab77c5c3e012"
server_key_version: "encryptionkey@0"
I20260809 06:28:35.645407 29391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "2229dfb877084b319ef5305479375118"
format_stamp: "Formatted at 2026-08-09 06:28:35 on dist-test-slave-mjjr"
server_key: "b5b51a73707aef87aeba4e731905ea8b"
server_key_iv: "904f564350b88a908442ab77c5c3e012"
server_key_version: "encryptionkey@0"
I20260809 06:28:35.651012 29391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:28:35.656291 29409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:35.658386 29391 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 06:28:35.658604 29391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "2229dfb877084b319ef5305479375118"
format_stamp: "Formatted at 2026-08-09 06:28:35 on dist-test-slave-mjjr"
server_key: "b5b51a73707aef87aeba4e731905ea8b"
server_key_iv: "904f564350b88a908442ab77c5c3e012"
server_key_version: "encryptionkey@0"
I20260809 06:28:35.658892 29391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:35.700670 29391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:35.704659 29391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:35.704908 29391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:35.723912 29391 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:45795
I20260809 06:28:35.723953 29483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:45795 every 8 connection(s)
I20260809 06:28:35.725328 29391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:28:35.727726 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 29391
I20260809 06:28:35.727948 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:28:35.728863 8716 external_mini_cluster.cc:1428] Setting key 9f9f30595a50c5ad84906459332fc0a1
I20260809 06:28:35.729658 29484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 06:28:35 dist-test-slave-mjjr krb5kdc[29371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:35.743921 29484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118: Bootstrap starting.
I20260809 06:28:35.747948 29484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:35.749311 29484 log.cc:824] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:35.753038 29484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118: No bootstrap required, opened a new log
I20260809 06:28:35.754017 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:35.730744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.8.131.62:45795)
0809 06:28:35.731553 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:35.731570 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:35.731622 (+ 52us) server_negotiation.cc:407] Connection header received
0809 06:28:35.733424 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:35.733450 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:35.733924 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:35.734349 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:35.735868 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:35.737961 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:35.739368 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:35.740469 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:35.740675 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:35.745003 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:35.745015 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:35.745032 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:35.745076 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:35.749071 (+ 3995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:35.749897 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:35.749908 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:35.749917 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:35.750244 (+ 327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:35.750807 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:35.750810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:35.750812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:35.751475 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:35.751813 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:35.752283 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 06:28:35.752713 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":322,"threads_started":1}
I20260809 06:28:35.758139 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:28:35.761813 29484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2229dfb877084b319ef5305479375118" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } }
I20260809 06:28:35.762181 29484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:35.762301 29484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2229dfb877084b319ef5305479375118, State: Initialized, Role: FOLLOWER
I20260809 06:28:35.762904 29484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2229dfb877084b319ef5305479375118" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } }
I20260809 06:28:35.763175 29484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:28:35.763350 29484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:28:35.763532 29484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:35.765769 29484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2229dfb877084b319ef5305479375118" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } }
I20260809 06:28:35.766274 29484 leader_election.cc:302] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2229dfb877084b319ef5305479375118; no voters:
I20260809 06:28:35.766861 29484 leader_election.cc:288] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:28:35.767045 29490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:28:35.767757 29490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [term 1 LEADER]: Becoming Leader. State: Replica: 2229dfb877084b319ef5305479375118, State: Running, Role: LEADER
I20260809 06:28:35.768265 29490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2229dfb877084b319ef5305479375118" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } }
I20260809 06:28:35.768960 29484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:28:35.771687 29492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2229dfb877084b319ef5305479375118. Latest consensus state: current_term: 1 leader_uuid: "2229dfb877084b319ef5305479375118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2229dfb877084b319ef5305479375118" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } } }
I20260809 06:28:35.772294 29492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:35.771709 29491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2229dfb877084b319ef5305479375118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2229dfb877084b319ef5305479375118" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 45795 } } }
I20260809 06:28:35.778162 29491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118 [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:35.778707 29499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:28:35.783056 29499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:28:35.791100 29499 catalog_manager.cc:1347] Generated new cluster ID: a9041d0f1d3745ce9384db59057c6c76
I20260809 06:28:35.791216 29499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:28:35.814335 29499 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:28:35.815758 29499 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:28:35.828151 29499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2229dfb877084b319ef5305479375118: Generated new TSK 0
I20260809 06:28:35.828980 29499 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:28:35.876691 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 29391
I20260809 06:28:35.878050 29478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:35.974690 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 29391
2026-08-09T06:28:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2326 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:28:36 dist-test-slave-mjjr krb5kdc[29522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:28:36 dist-test-slave-mjjr krb5kdc[29522](info): set up 2 sockets
Aug 09 06:28:36 dist-test-slave-mjjr krb5kdc[29522](info): commencing operation
krb5kdc: starting...
W20260809 06:28:37.271804 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.007s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:28:37 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:28:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:28:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:28:37.528630 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:39689
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:37.871192 29541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:37.871696 29541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:37.871881 29541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:37.880681 29541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:28:37.880815 29541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:37.880901 29541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:37.881000 29541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:28:37.881069 29541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:28:37.891518 29541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:39689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:37.892956 29541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:37.894301 29541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:37.906497 29546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:37.906450 29549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:37.906899 29552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:37.907267 29541 server_base.cc:1034] running on GCE node
I20260809 06:28:37.908138 29541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:37.909554 29541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:37.910825 29541 hybrid_clock.cc:648] HybridClock initialized: now 1786256917910772 us; error 53 us; skew 500 ppm
I20260809 06:28:37.911275 29541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:37 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256917, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:37.914463 29541 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:28:37.916043 29541 webserver.cc:460] Webserver started at http://127.8.131.62:38655/ using document root <none> and password file <none>
I20260809 06:28:37.916735 29541 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:37.916872 29541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:37.917204 29541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:28:37.920276 29541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "11c0d60f3f084aa0a150d6466a299bcb"
format_stamp: "Formatted at 2026-08-09 06:28:37 on dist-test-slave-mjjr"
server_key: "599389be7e885c300a3fa81b84c38503"
server_key_iv: "6f3d7b267b0b8d4b87b4a3b2e38e57ff"
server_key_version: "encryptionkey@0"
I20260809 06:28:37.921072 29541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "11c0d60f3f084aa0a150d6466a299bcb"
format_stamp: "Formatted at 2026-08-09 06:28:37 on dist-test-slave-mjjr"
server_key: "599389be7e885c300a3fa81b84c38503"
server_key_iv: "6f3d7b267b0b8d4b87b4a3b2e38e57ff"
server_key_version: "encryptionkey@0"
I20260809 06:28:37.926236 29541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:28:37.929689 29559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:37.930989 29541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:28:37.931178 29541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "11c0d60f3f084aa0a150d6466a299bcb"
format_stamp: "Formatted at 2026-08-09 06:28:37 on dist-test-slave-mjjr"
server_key: "599389be7e885c300a3fa81b84c38503"
server_key_iv: "6f3d7b267b0b8d4b87b4a3b2e38e57ff"
server_key_version: "encryptionkey@0"
I20260809 06:28:37.931375 29541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:37.947140 29541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:37.951232 29541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:37.951538 29541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:37.970887 29541 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:39689
I20260809 06:28:37.970929 29634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:39689 every 8 connection(s)
I20260809 06:28:37.972429 29541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:28:37.976162 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 29541
I20260809 06:28:37.976413 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:28:37.976558 29635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:37.977331 8716 external_mini_cluster.cc:1428] Setting key 73b9a39454a2761a20158231aee9af29
I20260809 06:28:37.991909 29635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Bootstrap starting.
Aug 09 06:28:37 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:37.995179 29635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:37.996366 29635 log.cc:824] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:37.999325 29635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: No bootstrap required, opened a new log
I20260809 06:28:38.001360 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:37.979490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44290 (local address 127.8.131.62:39689)
0809 06:28:37.980441 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:37.980452 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:37.980500 (+ 48us) server_negotiation.cc:407] Connection header received
0809 06:28:37.982014 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:37.982034 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:37.982419 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:37.982764 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:37.984620 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:37.985865 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:37.987779 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:37.988479 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:37.988613 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:37.993743 (+ 5130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:37.993751 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:37.993763 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:37.993807 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:37.997236 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:37.997994 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:37.998001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:37.998007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:37.998184 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:37.998649 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:37.998653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:37.998655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:37.999166 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:37.999544 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:37.999948 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 06:28:38.000363 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":497,"threads_started":1}
I20260809 06:28:38.007995 29635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:38.008399 29635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:38.008515 29635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c0d60f3f084aa0a150d6466a299bcb, State: Initialized, Role: FOLLOWER
I20260809 06:28:38.009177 29635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:38.009473 29635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:28:38.009652 29635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:28:38.009848 29635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:38.012423 29635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:38.012946 29635 leader_election.cc:302] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11c0d60f3f084aa0a150d6466a299bcb; no voters:
I20260809 06:28:38.013557 29635 leader_election.cc:288] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:28:38.013665 29641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:28:38.015075 29641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 1 LEADER]: Becoming Leader. State: Replica: 11c0d60f3f084aa0a150d6466a299bcb, State: Running, Role: LEADER
I20260809 06:28:38.015698 29641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:38.016129 29635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:28:38.018630 29642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11c0d60f3f084aa0a150d6466a299bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } } }
I20260809 06:28:38.018728 29645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11c0d60f3f084aa0a150d6466a299bcb. Latest consensus state: current_term: 1 leader_uuid: "11c0d60f3f084aa0a150d6466a299bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } } }
I20260809 06:28:38.019023 29642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:38.019155 29645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:38.022051 29652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:28:38.027323 29652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:28:38.036561 29652 catalog_manager.cc:1347] Generated new cluster ID: d90f199925ae494d9608e00c4dc8b496
I20260809 06:28:38.036695 29652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:28:38.047122 29652 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:28:38.049098 29652 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:28:38.058911 29652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Generated new TSK 0
I20260809 06:28:38.060091 29652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:28:38.136005 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:39689
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:38.478946 29673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:38.479259 29673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:38.479452 29673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:38.486783 29673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:38.486932 29673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:38.487106 29673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:28:38.496901 29673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:38.498189 29673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:38.499493 29673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:38.512403 29685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:38.512428 29688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:38.512518 29684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:38.513568 29673 server_base.cc:1034] running on GCE node
I20260809 06:28:38.514098 29673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:38.514935 29673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:38.516155 29673 hybrid_clock.cc:648] HybridClock initialized: now 1786256918516094 us; error 60 us; skew 500 ppm
I20260809 06:28:38.516444 29673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:38 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256918, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:38.519795 29673 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:28:38.521189 29673 webserver.cc:460] Webserver started at http://127.8.131.1:36799/ using document root <none> and password file <none>
I20260809 06:28:38.521879 29673 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:38.522037 29673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:38.522341 29673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:28:38.525444 29673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "97e987c4bda34cea887c2972d7916340"
format_stamp: "Formatted at 2026-08-09 06:28:38 on dist-test-slave-mjjr"
server_key: "f431ac96fdbb619008388a43b687c944"
server_key_iv: "c5ce194394ad76e853ef8a15bf4f112a"
server_key_version: "encryptionkey@0"
I20260809 06:28:38.526173 29673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "97e987c4bda34cea887c2972d7916340"
format_stamp: "Formatted at 2026-08-09 06:28:38 on dist-test-slave-mjjr"
server_key: "f431ac96fdbb619008388a43b687c944"
server_key_iv: "c5ce194394ad76e853ef8a15bf4f112a"
server_key_version: "encryptionkey@0"
I20260809 06:28:38.531442 29673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:28:38.534705 29695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:38.536199 29673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:28:38.536372 29673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "97e987c4bda34cea887c2972d7916340"
format_stamp: "Formatted at 2026-08-09 06:28:38 on dist-test-slave-mjjr"
server_key: "f431ac96fdbb619008388a43b687c944"
server_key_iv: "c5ce194394ad76e853ef8a15bf4f112a"
server_key_version: "encryptionkey@0"
I20260809 06:28:38.536577 29673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:38.577221 29673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:38.581254 29673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:38.581566 29673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:38.582553 29673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:38.583922 29673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:38.583995 29673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:38.584156 29673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:38.584218 29673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:38.618206 29673 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:45833
I20260809 06:28:38.618330 29847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:45833 every 8 connection(s)
I20260809 06:28:38.619504 29673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:28:38.627264 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 29673
I20260809 06:28:38.627489 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:28:38.628196 8716 external_mini_cluster.cc:1428] Setting key de1b86bcd7914bba2212a0699cade36e
Aug 09 06:28:38 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256918, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:38.641503 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:38.622012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:49711 (local address 127.8.131.62:39689)
0809 06:28:38.622499 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:38.622504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:38.623585 (+ 1081us) server_negotiation.cc:407] Connection header received
0809 06:28:38.625413 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:38.625418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:38.625512 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:38.625632 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:38.627123 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:38.628050 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:38.629793 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:38.630295 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:38.630438 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:38.634394 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:38.634403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:38.634411 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:38.634454 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:38.638241 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:38.639019 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:38.639027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:38.639033 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:38.639189 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:38.639604 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:38.639611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:38.639617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:38.640038 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:38.640194 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:38.640919 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 06:28:38.641185 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":257,"threads_started":1}
I20260809 06:28:38.642822 29852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:38.622581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:49711)
0809 06:28:38.623432 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:38.623479 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:38.625152 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:38.625861 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:38.625868 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:38.626350 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:38.626951 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:38.626968 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:38.628216 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:38.628220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:38.629636 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:38.629648 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:38.630702 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:38.630709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:38.630923 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:38.631706 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:38.631735 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:38.634164 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:38.638407 (+ 4243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:38.638415 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:38.638429 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:38.638877 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:38.639305 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:38.639308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:38.639310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:38.639494 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:38.640164 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:38.640169 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:38.640485 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 06:28:38.640770 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 06:28:38.641258 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":316,"threads_started":1}
I20260809 06:28:38.644433 29848 heartbeater.cc:347] Connected to a master server at 127.8.131.62:39689
I20260809 06:28:38.644752 29848 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:38.645466 29848 heartbeater.cc:511] Master 127.8.131.62:39689 requested a full tablet report, sending...
I20260809 06:28:38.647759 29584 ts_manager.cc:194] Registered new tserver with Master: 97e987c4bda34cea887c2972d7916340 (127.8.131.1:45833)
I20260809 06:28:38.649988 29584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:49711
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:28:38.688925 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:39689
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:39.033463 29856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:39.033767 29856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:39.033878 29856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:39.041698 29856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:39.041854 29856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:39.041986 29856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:28:39.052049 29856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:39.053382 29856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:39.054708 29856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:39.068848 29862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:39.068758 29865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:39.068810 29863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:39.069628 29856 server_base.cc:1034] running on GCE node
I20260809 06:28:39.070339 29856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:39.071357 29856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:39.072634 29856 hybrid_clock.cc:648] HybridClock initialized: now 1786256919072565 us; error 77 us; skew 500 ppm
I20260809 06:28:39.073000 29856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:39 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256919, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:39.076169 29856 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:28:39.077657 29856 webserver.cc:460] Webserver started at http://127.8.131.2:35857/ using document root <none> and password file <none>
I20260809 06:28:39.078358 29856 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:39.078513 29856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:39.078831 29856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:28:39.081776 29856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "de08f03f2ea146e795ea71ddeb3e6df8"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "f75bb010ba71ea3710777181c7067715"
server_key_iv: "66637460cb0a3b08e32975b27e430500"
server_key_version: "encryptionkey@0"
I20260809 06:28:39.082566 29856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "de08f03f2ea146e795ea71ddeb3e6df8"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "f75bb010ba71ea3710777181c7067715"
server_key_iv: "66637460cb0a3b08e32975b27e430500"
server_key_version: "encryptionkey@0"
I20260809 06:28:39.087831 29856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 06:28:39.091127 29873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:39.092433 29856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:28:39.092612 29856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "de08f03f2ea146e795ea71ddeb3e6df8"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "f75bb010ba71ea3710777181c7067715"
server_key_iv: "66637460cb0a3b08e32975b27e430500"
server_key_version: "encryptionkey@0"
I20260809 06:28:39.092803 29856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:39.115258 29856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:39.119753 29856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:39.120046 29856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:39.120991 29856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:39.122295 29856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:39.122421 29856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:39.122587 29856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:39.122659 29856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:39.156469 29856 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:37521
I20260809 06:28:39.156565 30029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:37521 every 8 connection(s)
I20260809 06:28:39.157750 29856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:28:39.165283 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 29856
I20260809 06:28:39.165472 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:28:39.165941 8716 external_mini_cluster.cc:1428] Setting key dd719a3a905bc01d3a5d5babed2c5d3f
Aug 09 06:28:39 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256919, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:39.178495 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.160247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:57947 (local address 127.8.131.62:39689)
0809 06:28:39.160748 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:39.160752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:39.161530 (+ 778us) server_negotiation.cc:407] Connection header received
0809 06:28:39.163367 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:39.163372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:39.163472 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:39.163600 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:39.165233 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.166124 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.167861 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.168490 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.168629 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.172606 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:39.172611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:39.172614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:39.172646 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:39.175291 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:39.176259 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:39.176263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:39.176265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:39.176390 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:39.176864 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:39.176867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:39.176869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:39.177111 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:39.177218 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:39.178072 (+ 854us) server_negotiation.cc:299] Negotiation successful
0809 06:28:39.178225 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":282,"threads_started":1}
I20260809 06:28:39.179579 30037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.160645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:57947)
0809 06:28:39.161347 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:39.161394 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:39.163086 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:39.163847 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:39.163854 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:39.164342 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:39.164960 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.164988 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.166274 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.166281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.167718 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.167730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.168702 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.168707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.168902 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.169591 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:39.169614 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:39.172373 (+ 2759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:39.175431 (+ 3058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:39.175441 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:39.175458 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:39.176144 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:39.176509 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:39.176516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:39.176522 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:39.176756 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:39.177241 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:39.177250 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:39.177680 (+ 430us) client_negotiation.cc:769] Sending connection context
0809 06:28:39.177994 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 06:28:39.178535 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":282,"threads_started":1}
I20260809 06:28:39.181051 30030 heartbeater.cc:347] Connected to a master server at 127.8.131.62:39689
I20260809 06:28:39.181377 30030 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:39.182102 30030 heartbeater.cc:511] Master 127.8.131.62:39689 requested a full tablet report, sending...
I20260809 06:28:39.183585 29584 ts_manager.cc:194] Registered new tserver with Master: de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521)
I20260809 06:28:39.184432 29584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:57947
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:28:39.224092 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:39689
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:39.556923 30041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:39.557227 30041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:39.557410 30041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:39.564344 30041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:39.564491 30041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:39.564677 30041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:28:39.574075 30041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:39.575342 30041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:39.576685 30041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:39.589867 30052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:39.589931 30054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:39.589954 30049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:39.590699 30041 server_base.cc:1034] running on GCE node
I20260809 06:28:39.591351 30041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:39.592365 30041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:39.593607 30041 hybrid_clock.cc:648] HybridClock initialized: now 1786256919593565 us; error 52 us; skew 500 ppm
I20260809 06:28:39.593951 30041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:39 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256919, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:39.597213 30041 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:28:39.598685 30041 webserver.cc:460] Webserver started at http://127.8.131.3:36363/ using document root <none> and password file <none>
I20260809 06:28:39.599354 30041 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:39.599494 30041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:39.599859 30041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:28:39.602707 30041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "23f8b23048bd4c5e8dfe2e580b0dd000"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "2baa576310022fce57dab0eaa1c10af4"
server_key_iv: "d74194a1070a35810d7baf90e3754784"
server_key_version: "encryptionkey@0"
I20260809 06:28:39.603483 30041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "23f8b23048bd4c5e8dfe2e580b0dd000"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "2baa576310022fce57dab0eaa1c10af4"
server_key_iv: "d74194a1070a35810d7baf90e3754784"
server_key_version: "encryptionkey@0"
I20260809 06:28:39.608727 30041 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:28:39.612162 30064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:39.613480 30041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:28:39.613668 30041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "23f8b23048bd4c5e8dfe2e580b0dd000"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "2baa576310022fce57dab0eaa1c10af4"
server_key_iv: "d74194a1070a35810d7baf90e3754784"
server_key_version: "encryptionkey@0"
I20260809 06:28:39.613895 30041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:39.640822 30041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:39.644812 30041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:39.645095 30041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:39.646037 30041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:39.647343 30041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:39.647445 30041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:39.647586 30041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:39.647675 30041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:39.653023 29848 heartbeater.cc:503] Master 127.8.131.62:39689 was elected leader, sending a full tablet report...
I20260809 06:28:39.681052 30041 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:42923
I20260809 06:28:39.681128 30235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:42923 every 8 connection(s)
I20260809 06:28:39.682345 30041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:28:39.689810 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 30041
I20260809 06:28:39.690042 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:28:39.690490 8716 external_mini_cluster.cc:1428] Setting key 01807d493a2805e47df09ac08beb20de
Aug 09 06:28:39 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256919, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:39.701925 30238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.684893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:41737 (local address 127.8.131.62:39689)
0809 06:28:39.685339 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:39.685344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:39.686283 (+ 939us) server_negotiation.cc:407] Connection header received
0809 06:28:39.688103 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:39.688107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:39.688175 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:39.688273 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:39.689842 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.690769 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.692573 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.693040 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.693195 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.696718 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:39.696723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:39.696726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:39.696757 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:39.699241 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:39.699980 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:39.699983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:39.699986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:39.700095 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:39.700462 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:39.700465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:39.700467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:39.700711 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:39.700826 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:39.701474 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 06:28:39.701628 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":197,"threads_started":1}
I20260809 06:28:39.702843 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.685433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:41737)
0809 06:28:39.686120 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:39.686163 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:39.687826 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:39.688465 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:39.688472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:39.688967 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:39.689660 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.689685 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.690932 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.690937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.692433 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.692449 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.693271 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.693280 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.693580 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.694248 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:39.694269 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:39.696507 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:39.699366 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:39.699372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:39.699383 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:39.699871 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:39.700188 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:39.700191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:39.700193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:39.700361 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:39.700820 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:39.700826 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:39.701115 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 06:28:39.701328 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 06:28:39.701733 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":279,"threads_started":1}
I20260809 06:28:39.704308 30236 heartbeater.cc:347] Connected to a master server at 127.8.131.62:39689
I20260809 06:28:39.704707 30236 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:39.705580 30236 heartbeater.cc:511] Master 127.8.131.62:39689 requested a full tablet report, sending...
I20260809 06:28:39.707018 29584 ts_manager.cc:194] Registered new tserver with Master: 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923)
I20260809 06:28:39.707818 29584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:41737
I20260809 06:28:39.718287 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:28:39.735924 30238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.723053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44292 (local address 127.8.131.62:39689)
0809 06:28:39.723271 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:39.723276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:39.723603 (+ 327us) server_negotiation.cc:407] Connection header received
0809 06:28:39.723844 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:39.723849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:39.723976 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:39.724104 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:39.725516 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.726251 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.727704 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.728130 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.728267 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.730653 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:39.730659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:39.730664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:39.730706 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:39.733209 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:39.733870 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:39.733875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:39.733879 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:39.734007 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:39.734381 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:39.734386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:39.734390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:39.734716 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:39.734978 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:39.735346 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 06:28:39.735581 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:28:39.739979 29584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:28:39.746850 29584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:28:39.784049 30255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.758527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42576)
0809 06:28:39.759532 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:39.759575 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:39.759761 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:39.776881 (+ 17120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:39.776887 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:39.776915 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:39.777240 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.777249 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.779187 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.779194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.781433 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.781448 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.782914 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.782920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.783163 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.783208 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 06:28:39.783405 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 06:28:39.783673 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":19561,"threads_started":1}
I20260809 06:28:39.784904 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.758875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.8.131.1:45833)
0809 06:28:39.776010 (+ 17135us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:39.776022 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:39.776064 (+ 42us) server_negotiation.cc:407] Connection header received
0809 06:28:39.776218 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:39.776230 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:39.776578 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:39.776873 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:39.777467 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.779045 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.781623 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.782668 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.782827 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.783100 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:39.784348 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0809 06:28:39.784538 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16909,"thread_start_us":16317,"threads_started":1}
I20260809 06:28:39.786501 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.758743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:42923 (local address 127.0.0.1:46968)
0809 06:28:39.774289 (+ 15546us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:39.774318 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:39.774502 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:39.778609 (+ 4107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:39.778617 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:39.778642 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:39.779018 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.779034 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.781001 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.781005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.783375 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.783393 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.785596 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.785602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.785782 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.785814 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 06:28:39.785922 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 06:28:39.786056 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15351,"mutex_wait_us":2,"thread_start_us":430,"threads_started":1}
I20260809 06:28:39.787215 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.759275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.8.131.3:42923)
0809 06:28:39.777989 (+ 18714us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:39.777997 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:39.778032 (+ 35us) server_negotiation.cc:407] Connection header received
0809 06:28:39.778125 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:39.778132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:39.778354 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:39.778552 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:39.779246 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.780824 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.783617 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.785353 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.785506 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.785744 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:39.785989 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 06:28:39.786891 (+ 902us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18519,"thread_start_us":18368,"threads_started":1}
I20260809 06:28:39.787246 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.759975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:37521 (local address 127.0.0.1:58824)
0809 06:28:39.778362 (+ 18387us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:39.778388 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:39.778548 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:39.779456 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:39.779462 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:39.779493 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:39.780066 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.780080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.781858 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.781864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.784327 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.784343 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.786312 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.786319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.786549 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.786587 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 06:28:39.786680 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 06:28:39.786831 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18259,"mutex_wait_us":67,"thread_start_us":14648,"threads_started":1}
I20260809 06:28:39.787608 30260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.760327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58824 (local address 127.8.131.2:37521)
0809 06:28:39.775533 (+ 15206us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:39.775544 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:39.778513 (+ 2969us) server_negotiation.cc:407] Connection header received
0809 06:28:39.778758 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:39.778768 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:39.778999 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:39.779189 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:39.780262 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.781704 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.784509 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.785991 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.786115 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.786308 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:39.786861 (+ 553us) server_negotiation.cc:299] Negotiation successful
0809 06:28:39.787050 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15012,"thread_start_us":15204,"threads_started":1}
I20260809 06:28:39.789355 29765 tablet_service.cc:1467] Processing CreateTablet for tablet fb48ddd425f84b8b85b31eb0ac7a3b29 (DEFAULT_TABLE table=test-table [id=134f0ba04b6f44089841d3778410a313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:28:39.790742 29765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb48ddd425f84b8b85b31eb0ac7a3b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:39.790901 29941 tablet_service.cc:1467] Processing CreateTablet for tablet fb48ddd425f84b8b85b31eb0ac7a3b29 (DEFAULT_TABLE table=test-table [id=134f0ba04b6f44089841d3778410a313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:28:39.790901 30135 tablet_service.cc:1467] Processing CreateTablet for tablet fb48ddd425f84b8b85b31eb0ac7a3b29 (DEFAULT_TABLE table=test-table [id=134f0ba04b6f44089841d3778410a313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:28:39.792284 29941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb48ddd425f84b8b85b31eb0ac7a3b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:39.792284 30135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb48ddd425f84b8b85b31eb0ac7a3b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:39.802737 30261 tablet_bootstrap.cc:492] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: Bootstrap starting.
I20260809 06:28:39.806420 30261 tablet_bootstrap.cc:654] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:39.807229 30262 tablet_bootstrap.cc:492] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: Bootstrap starting.
I20260809 06:28:39.808077 30263 tablet_bootstrap.cc:492] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: Bootstrap starting.
I20260809 06:28:39.809131 30261 log.cc:824] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:39.811059 30262 tablet_bootstrap.cc:654] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:39.811211 30263 tablet_bootstrap.cc:654] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:39.812275 30263 log.cc:824] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:39.812433 30262 log.cc:824] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:39.812812 30261 tablet_bootstrap.cc:492] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: No bootstrap required, opened a new log
I20260809 06:28:39.813174 30261 ts_tablet_manager.cc:1397] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 06:28:39.817888 30262 tablet_bootstrap.cc:492] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: No bootstrap required, opened a new log
I20260809 06:28:39.818043 30263 tablet_bootstrap.cc:492] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: No bootstrap required, opened a new log
I20260809 06:28:39.818212 30262 ts_tablet_manager.cc:1397] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 06:28:39.818346 30263 ts_tablet_manager.cc:1397] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20260809 06:28:39.825227 30261 raft_consensus.cc:348] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } }
I20260809 06:28:39.825742 30261 raft_consensus.cc:374] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:39.825863 30261 raft_consensus.cc:729] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e987c4bda34cea887c2972d7916340, State: Initialized, Role: FOLLOWER
I20260809 06:28:39.826550 30261 consensus_queue.cc:260] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } }
I20260809 06:28:39.826690 30262 raft_consensus.cc:348] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } }
I20260809 06:28:39.827078 30262 raft_consensus.cc:374] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:39.827189 30262 raft_consensus.cc:729] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de08f03f2ea146e795ea71ddeb3e6df8, State: Initialized, Role: FOLLOWER
I20260809 06:28:39.827796 30262 consensus_queue.cc:260] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } }
I20260809 06:28:39.828804 30261 ts_tablet_manager.cc:1428] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260809 06:28:39.828872 30030 heartbeater.cc:503] Master 127.8.131.62:39689 was elected leader, sending a full tablet report...
I20260809 06:28:39.829473 30262 ts_tablet_manager.cc:1428] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260809 06:28:39.830837 30263 raft_consensus.cc:348] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } }
I20260809 06:28:39.831283 30263 raft_consensus.cc:374] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:39.831398 30263 raft_consensus.cc:729] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f8b23048bd4c5e8dfe2e580b0dd000, State: Initialized, Role: FOLLOWER
I20260809 06:28:39.832056 30263 consensus_queue.cc:260] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } }
I20260809 06:28:39.833181 30236 heartbeater.cc:503] Master 127.8.131.62:39689 was elected leader, sending a full tablet report...
I20260809 06:28:39.835245 30263 ts_tablet_manager.cc:1428] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
W20260809 06:28:39.872113 29850 tablet.cc:2369] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:28:39.909796 30032 tablet.cc:2369] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:28:39.934334 30237 tablet.cc:2369] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:28:39.937232 30271 raft_consensus.cc:482] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:28:39.937562 30271 raft_consensus.cc:504] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } }
I20260809 06:28:39.939229 30271 leader_election.cc:288] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923), de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521)
I20260809 06:28:39.946493 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.939603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:60723 (local address 127.8.131.3:42923)
0809 06:28:39.939805 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:39.939810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:39.940420 (+ 610us) server_negotiation.cc:407] Connection header received
0809 06:28:39.940669 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:39.940676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:39.940777 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:39.940883 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:39.941640 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.942608 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.944663 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.945469 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.945568 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.945645 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:39.946071 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 06:28:39.946207 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 06:28:39.946537 30260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.940696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:59849 (local address 127.8.131.2:37521)
0809 06:28:39.940911 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:39.940917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:39.941215 (+ 298us) server_negotiation.cc:407] Connection header received
0809 06:28:39.941358 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:39.941365 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:39.941446 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:39.941548 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:39.942181 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.942982 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.944791 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.945585 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.945742 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.945839 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:39.946143 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 06:28:39.946283 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 06:28:39.946627 30274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.939528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:42923 (local address 127.8.131.1:60723)
0809 06:28:39.940308 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:39.940328 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:39.940477 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:39.941006 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:39.941013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:39.941049 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:39.941486 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.941501 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.942739 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.942744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.944493 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.944505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.945684 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.945692 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.945886 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.945926 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 06:28:39.946002 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 06:28:39.946133 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":39,"spinlock_wait_cycles":33152,"thread_start_us":428,"threads_started":1}
I20260809 06:28:39.946692 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:39.940402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:37521 (local address 127.8.131.1:59849)
0809 06:28:39.941092 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:39.941113 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:39.941237 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:39.941582 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:39.941587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:39.941612 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:39.941999 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.942030 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.943125 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.943129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.944633 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:39.944648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:39.945817 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:39.945822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:39.945952 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:39.945974 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 06:28:39.946095 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 06:28:39.946214 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":1,"thread_start_us":355,"threads_started":1}
I20260809 06:28:39.947705 30168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb48ddd425f84b8b85b31eb0ac7a3b29" candidate_uuid: "97e987c4bda34cea887c2972d7916340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" is_pre_election: true
I20260809 06:28:39.947767 29966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb48ddd425f84b8b85b31eb0ac7a3b29" candidate_uuid: "97e987c4bda34cea887c2972d7916340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" is_pre_election: true
I20260809 06:28:39.948283 30168 raft_consensus.cc:2454] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e987c4bda34cea887c2972d7916340 in term 0.
I20260809 06:28:39.948351 29966 raft_consensus.cc:2454] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e987c4bda34cea887c2972d7916340 in term 0.
I20260809 06:28:39.949052 29697 leader_election.cc:302] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97e987c4bda34cea887c2972d7916340, de08f03f2ea146e795ea71ddeb3e6df8; no voters:
I20260809 06:28:39.949458 30271 raft_consensus.cc:2781] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:28:39.949630 30271 raft_consensus.cc:482] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:28:39.949740 30271 raft_consensus.cc:3037] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:39.952040 30271 raft_consensus.cc:504] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } }
I20260809 06:28:39.952636 30271 leader_election.cc:288] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 1 election: Requested vote from peers 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923), de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521)
I20260809 06:28:39.953138 30168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb48ddd425f84b8b85b31eb0ac7a3b29" candidate_uuid: "97e987c4bda34cea887c2972d7916340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000"
I20260809 06:28:39.953219 29966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb48ddd425f84b8b85b31eb0ac7a3b29" candidate_uuid: "97e987c4bda34cea887c2972d7916340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de08f03f2ea146e795ea71ddeb3e6df8"
I20260809 06:28:39.953348 30168 raft_consensus.cc:3037] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:39.953467 29966 raft_consensus.cc:3037] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:39.955608 30168 raft_consensus.cc:2454] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e987c4bda34cea887c2972d7916340 in term 1.
I20260809 06:28:39.955740 29966 raft_consensus.cc:2454] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e987c4bda34cea887c2972d7916340 in term 1.
I20260809 06:28:39.956076 29697 leader_election.cc:302] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f8b23048bd4c5e8dfe2e580b0dd000, 97e987c4bda34cea887c2972d7916340; no voters:
I20260809 06:28:39.956377 30271 raft_consensus.cc:2781] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:28:39.956822 30271 raft_consensus.cc:686] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 1 LEADER]: Becoming Leader. State: Replica: 97e987c4bda34cea887c2972d7916340, State: Running, Role: LEADER
I20260809 06:28:39.957651 30271 consensus_queue.cc:237] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } }
I20260809 06:28:39.962390 29584 catalog_manager.cc:5495] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97e987c4bda34cea887c2972d7916340 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "97e987c4bda34cea887c2972d7916340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } health_report { overall_health: UNKNOWN } } }
I20260809 06:28:40.007157 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:40.001773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42578 (local address 127.8.131.1:45833)
0809 06:28:40.002021 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:40.002027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:40.002045 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:28:40.002142 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:40.002145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:40.002232 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:40.002407 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:28:40.002792 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:40.003528 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:40.004949 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:40.005364 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:40.005509 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:40.006056 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:28:40.006585 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:28:40.006754 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:40.006808 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 06:28:40.006891 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:28:40.015467 30168 raft_consensus.cc:1261] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 1 FOLLOWER]: Refusing update from remote peer 97e987c4bda34cea887c2972d7916340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:28:40.015590 29966 raft_consensus.cc:1261] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [term 1 FOLLOWER]: Refusing update from remote peer 97e987c4bda34cea887c2972d7916340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:28:40.016547 30271 consensus_queue.cc:1035] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:28:40.016876 30276 consensus_queue.cc:1035] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:28:40.028216 30280 mvcc.cc:205] Tried to move back new op lower bound from 7316508344372359168 to 7316508344150351872. Current Snapshot: MvccSnapshot[applied={T|T < 7316508344372359168}]
I20260809 06:28:40.043087 29584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:28:40.043358 29584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:28:40.046773 29584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Sending DeleteTablet for 3 replicas of tablet fb48ddd425f84b8b85b31eb0ac7a3b29
I20260809 06:28:40.047966 29941 tablet_service.cc:1514] Processing DeleteTablet for tablet fb48ddd425f84b8b85b31eb0ac7a3b29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:28:40 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:58824
I20260809 06:28:40.047962 29765 tablet_service.cc:1514] Processing DeleteTablet for tablet fb48ddd425f84b8b85b31eb0ac7a3b29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:28:40 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:42576
I20260809 06:28:40.048327 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 29673
I20260809 06:28:40.048978 30135 tablet_service.cc:1514] Processing DeleteTablet for tablet fb48ddd425f84b8b85b31eb0ac7a3b29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:28:40 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:46968
I20260809 06:28:40.049310 30299 tablet_replica.cc:330] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: stopping tablet replica
I20260809 06:28:40.049750 30299 raft_consensus.cc:2229] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:28:40.050277 30299 raft_consensus.cc:2258] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:28:40.051064 30298 tablet_replica.cc:330] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: stopping tablet replica
I20260809 06:28:40.051445 30298 raft_consensus.cc:2229] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:28:40.052126 30298 raft_consensus.cc:2258] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:28:40.053483 30300 tablet_replica.cc:330] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: stopping tablet replica
I20260809 06:28:40.053869 30300 raft_consensus.cc:2229] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:28:40.054348 30300 raft_consensus.cc:2258] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:28:40.055176 30299 ts_tablet_manager.cc:1905] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:28:40.056545 30298 ts_tablet_manager.cc:1905] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:28:40.062549 30299 ts_tablet_manager.cc:1918] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:28:40.062762 30299 log.cc:1196] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/fb48ddd425f84b8b85b31eb0ac7a3b29
I20260809 06:28:40.063370 30299 ts_tablet_manager.cc:1939] T fb48ddd425f84b8b85b31eb0ac7a3b29 P de08f03f2ea146e795ea71ddeb3e6df8: Deleting consensus metadata
Aug 09 06:28:40 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:28:40.065516 29562 catalog_manager.cc:4841] TS de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521): tablet fb48ddd425f84b8b85b31eb0ac7a3b29 (table test-table [id=134f0ba04b6f44089841d3778410a313]) successfully deleted
I20260809 06:28:40.066008 30300 ts_tablet_manager.cc:1905] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:28:40.071235 30298 ts_tablet_manager.cc:1918] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:28:40.071437 30298 log.cc:1196] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/fb48ddd425f84b8b85b31eb0ac7a3b29
I20260809 06:28:40.072057 30298 ts_tablet_manager.cc:1939] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 97e987c4bda34cea887c2972d7916340: Deleting consensus metadata
I20260809 06:28:40.073133 30300 ts_tablet_manager.cc:1918] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:28:40.073299 30300 log.cc:1196] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/fb48ddd425f84b8b85b31eb0ac7a3b29
I20260809 06:28:40.073858 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:40.052787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.8.131.1:45833)
0809 06:28:40.052994 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:40.053003 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:40.053029 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:28:40.053097 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:40.053104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:40.053194 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:40.053316 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:40.055734 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:40.057231 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:40.059373 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:40.060209 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:40.060305 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:40.066643 (+ 6338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:40.066656 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:40.066670 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:40.066710 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:40.069878 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:40.070600 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:40.070609 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:40.070613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:40.070809 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:40.071292 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:40.071301 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:40.071304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:40.072048 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:40.072426 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:40.073046 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 06:28:40.073454 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 06:28:40.073961 29560 catalog_manager.cc:4841] TS 97e987c4bda34cea887c2972d7916340 (127.8.131.1:45833): tablet fb48ddd425f84b8b85b31eb0ac7a3b29 (table test-table [id=134f0ba04b6f44089841d3778410a313]) successfully deleted
I20260809 06:28:40.074473 30300 ts_tablet_manager.cc:1939] T fb48ddd425f84b8b85b31eb0ac7a3b29 P 23f8b23048bd4c5e8dfe2e580b0dd000: Deleting consensus metadata
I20260809 06:28:40.075115 29842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:40.076241 29562 catalog_manager.cc:4841] TS 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923): tablet fb48ddd425f84b8b85b31eb0ac7a3b29 (table test-table [id=134f0ba04b6f44089841d3778410a313]) successfully deleted
I20260809 06:28:40.174705 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 29673
I20260809 06:28:40.196470 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 29856
Aug 09 06:28:40 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:28:40.216398 30260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:40.197252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58826 (local address 127.8.131.2:37521)
0809 06:28:40.197420 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:40.197424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:40.197464 (+ 40us) server_negotiation.cc:407] Connection header received
0809 06:28:40.197629 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:40.197633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:40.197729 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:40.197808 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:40.199147 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:40.199970 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:40.201407 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:40.201778 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:40.201868 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:40.210661 (+ 8793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:40.210675 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:40.210690 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:40.210727 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:40.213768 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:40.214483 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:40.214489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:40.214492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:40.214637 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:40.214951 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:40.214954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:40.214956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:40.215330 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:40.215554 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:40.215973 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 06:28:40.216136 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 06:28:40.217094 30022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:40.308641 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 29856
I20260809 06:28:40.330855 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 30041
Aug 09 06:28:40 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:28:40.345359 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:40.331687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46984 (local address 127.8.131.3:42923)
0809 06:28:40.331882 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:40.331886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:40.331905 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:28:40.332043 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:40.332047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:40.332121 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:40.332204 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:40.333491 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:40.334205 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:40.335510 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:40.336152 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:40.336246 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:40.340365 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:40.340381 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:40.340397 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:40.340429 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:40.342963 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:40.343556 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:40.343562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:40.343564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:40.343739 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:40.344059 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:40.344062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:40.344064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:40.344384 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:40.344616 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:40.344855 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 06:28:40.345009 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 06:28:40.346067 30229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:40.434564 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 30041
I20260809 06:28:40.457509 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 29541
I20260809 06:28:40.458734 29627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:40.570847 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 29541
I20260809 06:28:40.601066 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:39689
--webserver_interface=127.8.131.62
--webserver_port=38655
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:40.943702 30307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:40.943994 30307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:40.944178 30307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:40.951356 30307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:28:40.951455 30307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:40.951502 30307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:40.951592 30307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:28:40.951686 30307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:28:40.961560 30307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:39689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=38655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:40.962877 30307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:40.964282 30307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:40.990993 30317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:40.991042 30315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:40.990962 30314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:40.996917 30307 server_base.cc:1034] running on GCE node
I20260809 06:28:40.997838 30307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:40.999496 30307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:41.000713 30307 hybrid_clock.cc:648] HybridClock initialized: now 1786256921000653 us; error 73 us; skew 500 ppm
I20260809 06:28:41.001168 30307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:41 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256921, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:41.009444 30307 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:28:41.011359 30307 webserver.cc:460] Webserver started at http://127.8.131.62:38655/ using document root <none> and password file <none>
I20260809 06:28:41.012212 30307 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:41.012336 30307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:41.018405 30307 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 06:28:41.021616 30325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:41.022981 30307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:28:41.023138 30307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "11c0d60f3f084aa0a150d6466a299bcb"
format_stamp: "Formatted at 2026-08-09 06:28:37 on dist-test-slave-mjjr"
server_key: "599389be7e885c300a3fa81b84c38503"
server_key_iv: "6f3d7b267b0b8d4b87b4a3b2e38e57ff"
server_key_version: "encryptionkey@0"
I20260809 06:28:41.024282 30307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:41.049566 30307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:41.054095 30307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:41.054366 30307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:41.075500 30307 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:39689
I20260809 06:28:41.075519 30392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:39689 every 8 connection(s)
I20260809 06:28:41.077309 30307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:28:41.079532 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 30307
I20260809 06:28:41.080632 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:45833
--local_ip_for_outbound_sockets=127.8.131.1
--tserver_master_addrs=127.8.131.62:39689
--webserver_port=36799
--webserver_interface=127.8.131.1
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:28:41.087531 30396 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:28:41.092609 30396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Bootstrap starting.
I20260809 06:28:41.112385 30396 log.cc:824] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:41.124861 30396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:28:41.125427 30396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Bootstrap complete.
I20260809 06:28:41.135174 30396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:41.136071 30396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c0d60f3f084aa0a150d6466a299bcb, State: Initialized, Role: FOLLOWER
I20260809 06:28:41.136804 30396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:41.137059 30396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:28:41.137178 30396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:28:41.137379 30396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:28:41.141294 30396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:41.141837 30396 leader_election.cc:302] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11c0d60f3f084aa0a150d6466a299bcb; no voters:
I20260809 06:28:41.142549 30396 leader_election.cc:288] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 06:28:41.142679 30402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 06:28:41.143976 30402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 2 LEADER]: Becoming Leader. State: Replica: 11c0d60f3f084aa0a150d6466a299bcb, State: Running, Role: LEADER
I20260809 06:28:41.144565 30402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:41.144984 30396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:28:41.147219 30404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "11c0d60f3f084aa0a150d6466a299bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } } }
I20260809 06:28:41.147716 30404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:41.147941 30405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11c0d60f3f084aa0a150d6466a299bcb. Latest consensus state: current_term: 2 leader_uuid: "11c0d60f3f084aa0a150d6466a299bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } } }
I20260809 06:28:41.148416 30405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:41.153214 30411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:28:41.161989 30411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:28:41.163512 30411 catalog_manager.cc:1259] Loaded cluster ID: d90f199925ae494d9608e00c4dc8b496
I20260809 06:28:41.163609 30411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:28:41.168035 30411 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 06:28:41.189831 30411 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 06:28:41.197013 30411 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:28:41.198390 30411 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Loaded TSK: 0
I20260809 06:28:41.199498 30411 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 06:28:41.448927 30398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:41.449219 30398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:41.449381 30398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:41.457134 30398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:41.457283 30398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:41.457470 30398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:28:41.466962 30398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:45833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=36799
--tserver_master_addrs=127.8.131.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:41.468348 30398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:41.469663 30398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:41.484131 30439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:41.484753 30442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:41.484866 30440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:41.485592 30398 server_base.cc:1034] running on GCE node
I20260809 06:28:41.486244 30398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:41.487262 30398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:41.488482 30398 hybrid_clock.cc:648] HybridClock initialized: now 1786256921488428 us; error 46 us; skew 500 ppm
I20260809 06:28:41.488792 30398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:41 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256921, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:41.492031 30398 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:28:41.493571 30398 webserver.cc:460] Webserver started at http://127.8.131.1:36799/ using document root <none> and password file <none>
I20260809 06:28:41.494259 30398 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:41.494398 30398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:41.499800 30398 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:28:41.502527 30452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:41.503976 30398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:28:41.504149 30398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "97e987c4bda34cea887c2972d7916340"
format_stamp: "Formatted at 2026-08-09 06:28:38 on dist-test-slave-mjjr"
server_key: "f431ac96fdbb619008388a43b687c944"
server_key_iv: "c5ce194394ad76e853ef8a15bf4f112a"
server_key_version: "encryptionkey@0"
I20260809 06:28:41.505064 30398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:41.517755 30398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:41.522357 30398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:41.522639 30398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:41.523730 30398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:41.525094 30398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:41.525182 30398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:41.525326 30398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:41.525430 30398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:41.563514 30398 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:45833
I20260809 06:28:41.563611 30606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:45833 every 8 connection(s)
I20260809 06:28:41.565037 30398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:28:41.569702 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 30398
I20260809 06:28:41.570358 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:37521
--local_ip_for_outbound_sockets=127.8.131.2
--tserver_master_addrs=127.8.131.62:39689
--webserver_port=35857
--webserver_interface=127.8.131.2
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:28:41 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256921, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:41.589920 30610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:41.567863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:43847 (local address 127.8.131.62:39689)
0809 06:28:41.568486 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:41.568498 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:41.569073 (+ 575us) server_negotiation.cc:407] Connection header received
0809 06:28:41.571374 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:41.571394 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:41.571878 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:41.572269 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:41.573656 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:41.575684 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:41.577174 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:41.578233 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:41.578435 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:41.582512 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:41.582519 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:41.582532 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:41.582574 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:41.585532 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:41.586455 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:41.586471 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:41.586476 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:41.586632 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:41.587151 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:41.587154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:41.587156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:41.587592 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:41.587809 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:41.588696 (+ 887us) server_negotiation.cc:299] Negotiation successful
0809 06:28:41.589011 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":288,"threads_started":1}
I20260809 06:28:41.590435 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:41.568068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:43847)
0809 06:28:41.568896 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:41.568950 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:41.570818 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:41.572364 (+ 1546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:41.572371 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:41.572865 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:41.573405 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:41.573422 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:41.575830 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:41.575834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:41.577018 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:41.577035 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:41.578450 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:41.578455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:41.578640 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:41.579327 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:41.579349 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:41.582244 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:41.585669 (+ 3425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:41.585681 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:41.585699 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:41.586316 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:41.586773 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:41.586781 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:41.586788 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:41.587041 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:41.587779 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:41.587791 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:41.588210 (+ 419us) client_negotiation.cc:769] Sending connection context
0809 06:28:41.588531 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 06:28:41.589124 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":363,"threads_started":1}
I20260809 06:28:41.593245 30607 heartbeater.cc:347] Connected to a master server at 127.8.131.62:39689
I20260809 06:28:41.593608 30607 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:41.594599 30607 heartbeater.cc:511] Master 127.8.131.62:39689 requested a full tablet report, sending...
I20260809 06:28:41.597050 30348 ts_manager.cc:194] Registered new tserver with Master: 97e987c4bda34cea887c2972d7916340 (127.8.131.1:45833)
I20260809 06:28:41.598577 30348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:43847
W20260809 06:28:41.909713 30612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:41.909993 30612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:41.910184 30612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:41.917369 30612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:41.917516 30612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:41.917708 30612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:28:41.927627 30612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:37521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=35857
--tserver_master_addrs=127.8.131.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:41.929052 30612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:41.930327 30612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:41.943461 30619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:41.943483 30618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:41.944178 30622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:41.944152 30612 server_base.cc:1034] running on GCE node
I20260809 06:28:41.944869 30612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:41.945832 30612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:41.947063 30612 hybrid_clock.cc:648] HybridClock initialized: now 1786256921947029 us; error 50 us; skew 500 ppm
I20260809 06:28:41.947369 30612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:41 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256921, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:41.950482 30612 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:28:41.951987 30612 webserver.cc:460] Webserver started at http://127.8.131.2:35857/ using document root <none> and password file <none>
I20260809 06:28:41.952663 30612 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:41.952819 30612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:41.958225 30612 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 06:28:41.961018 30631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:41.962306 30612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 06:28:41.962520 30612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "de08f03f2ea146e795ea71ddeb3e6df8"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "f75bb010ba71ea3710777181c7067715"
server_key_iv: "66637460cb0a3b08e32975b27e430500"
server_key_version: "encryptionkey@0"
I20260809 06:28:41.963482 30612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:41.978396 30612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:41.982298 30612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:41.982594 30612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:41.983537 30612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:41.984854 30612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:41.984984 30612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:41.985118 30612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:41.985206 30612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:42.019034 30612 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:37521
I20260809 06:28:42.019100 30788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:37521 every 8 connection(s)
I20260809 06:28:42.020853 30612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:28:42.030133 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 30612
I20260809 06:28:42.030977 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:42923
--local_ip_for_outbound_sockets=127.8.131.3
--tserver_master_addrs=127.8.131.62:39689
--webserver_port=36363
--webserver_interface=127.8.131.3
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:28:42 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256921, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:42.041560 30610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.023374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:50609 (local address 127.8.131.62:39689)
0809 06:28:42.023553 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.023557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.024679 (+ 1122us) server_negotiation.cc:407] Connection header received
0809 06:28:42.026374 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.026378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.026454 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.026579 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:42.028099 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.028887 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.030468 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.031197 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.031347 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.034831 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:42.034840 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:42.034847 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:42.034892 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:42.038464 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:42.039168 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:42.039176 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:42.039183 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:42.039343 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:42.039795 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:42.039802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:42.039807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:42.040191 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:42.040348 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.041018 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.041291 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:28:42.042816 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.023811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:50609)
0809 06:28:42.024513 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:42.024550 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:42.026135 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:42.026770 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:42.026777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:42.027252 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:42.027945 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.027963 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.029060 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.029064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.030298 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.030315 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.031413 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.031418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.031623 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.032392 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:42.032415 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:42.034610 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:42.038605 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:42.038611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:42.038623 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:42.039033 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:42.039450 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:42.039453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:42.039456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:42.039625 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:42.040313 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:42.040318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:42.040627 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 06:28:42.040846 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 06:28:42.041213 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":269,"threads_started":1}
I20260809 06:28:42.044433 30790 heartbeater.cc:347] Connected to a master server at 127.8.131.62:39689
I20260809 06:28:42.044765 30790 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:42.045497 30790 heartbeater.cc:511] Master 127.8.131.62:39689 requested a full tablet report, sending...
I20260809 06:28:42.047070 30348 ts_manager.cc:194] Registered new tserver with Master: de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521)
I20260809 06:28:42.048346 30348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:50609
W20260809 06:28:42.384145 30795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:42.384392 30795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:42.384536 30795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:42.391880 30795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:42.391978 30795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:42.392180 30795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:28:42.401633 30795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:42923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=36363
--tserver_master_addrs=127.8.131.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:42.402863 30795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:42.404201 30795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:42.420549 30803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:42.421373 30802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:42.421789 30807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:42.422151 30795 server_base.cc:1034] running on GCE node
I20260809 06:28:42.422883 30795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:42.423991 30795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:42.425277 30795 hybrid_clock.cc:648] HybridClock initialized: now 1786256922425222 us; error 46 us; skew 500 ppm
I20260809 06:28:42.425621 30795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:42 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256922, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:42.428889 30795 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:28:42.430374 30795 webserver.cc:460] Webserver started at http://127.8.131.3:36363/ using document root <none> and password file <none>
I20260809 06:28:42.431072 30795 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:42.431226 30795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:42.437427 30795 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 06:28:42.440347 30815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:42.442070 30795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:28:42.442247 30795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "23f8b23048bd4c5e8dfe2e580b0dd000"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "2baa576310022fce57dab0eaa1c10af4"
server_key_iv: "d74194a1070a35810d7baf90e3754784"
server_key_version: "encryptionkey@0"
I20260809 06:28:42.443202 30795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:42.469332 30795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:42.473728 30795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:42.474017 30795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:42.475066 30795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:42.476531 30795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:42.476670 30795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:42.476788 30795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:42.476876 30795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:42.513077 30795 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:42923
I20260809 06:28:42.513160 30977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:42923 every 8 connection(s)
I20260809 06:28:42.514482 30795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:28:42.521670 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 30795
Aug 09 06:28:42 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256922, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:28:42.535614 30610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.516936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:46111 (local address 127.8.131.62:39689)
0809 06:28:42.517110 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.517114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.518180 (+ 1066us) server_negotiation.cc:407] Connection header received
0809 06:28:42.520082 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.520087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.520184 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.520286 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:42.522111 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.523273 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.525013 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.525445 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.525622 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.529385 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:42.529391 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:42.529398 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:42.529433 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:42.532337 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:42.533255 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:42.533260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:42.533270 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:42.533414 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:42.533910 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:42.533914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:42.533916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:42.534209 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:42.534365 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.535065 (+ 700us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.535278 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 06:28:42.536640 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.522615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44308 (local address 127.8.131.62:39689)
0809 06:28:42.523106 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.523113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.523421 (+ 308us) server_negotiation.cc:407] Connection header received
0809 06:28:42.523607 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.523615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.523726 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.523837 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:42.525382 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.526186 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.527556 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.528343 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.528483 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.531089 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:42.531095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:42.531105 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:42.531147 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:42.533910 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:42.534622 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:42.534627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:42.534630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:42.534750 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:42.535162 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:42.535167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:42.535171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:42.535455 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:42.535741 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.536054 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.536267 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":248,"threads_started":1}
I20260809 06:28:42.537032 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.517312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:46111)
0809 06:28:42.518006 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:42.518048 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:42.519798 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:42.520511 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:42.520556 (+ 45us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:42.521122 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:42.521940 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.521966 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.523435 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.523440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.524860 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.524872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.525694 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.525703 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.525946 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.526629 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:42.526651 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:42.529157 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:42.532515 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:42.532523 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:42.532539 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:42.533120 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:42.533535 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:42.533543 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:42.533546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:42.533738 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:42.534351 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:42.534358 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:42.534689 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 06:28:42.534954 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 06:28:42.535471 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":279,"threads_started":1}
I20260809 06:28:42.538487 30978 heartbeater.cc:347] Connected to a master server at 127.8.131.62:39689
I20260809 06:28:42.538782 30978 heartbeater.cc:464] Registering TS with master...
I20260809 06:28:42.539556 30978 heartbeater.cc:511] Master 127.8.131.62:39689 requested a full tablet report, sending...
I20260809 06:28:42.540871 30348 ts_manager.cc:194] Registered new tserver with Master: 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923)
I20260809 06:28:42.541677 30348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:46111
I20260809 06:28:42.551460 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:28:42.558597 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.552496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.8.131.62:39689)
0809 06:28:42.552702 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.552711 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.553088 (+ 377us) server_negotiation.cc:407] Connection header received
0809 06:28:42.553343 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.553346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.553480 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.553633 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:28:42.554107 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.554798 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.556435 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.556855 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.556998 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.557514 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:28:42.557993 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:28:42.558148 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.558227 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.558308 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 06:28:42.559475 30348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:28:42.566896 30348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:28:42.589210 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.578151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42604)
0809 06:28:42.579778 (+ 1627us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:42.579855 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:42.580093 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:42.581284 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:42.581295 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:42.581342 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:42.581884 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.581898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.583964 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.583968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.586442 (+ 2474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.586455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.588169 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.588176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.588370 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.588441 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 06:28:42.588595 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 06:28:42.588778 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1170,"mutex_wait_us":24,"thread_start_us":1043,"threads_started":1}
I20260809 06:28:42.590145 30999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.578362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.8.131.1:45833)
0809 06:28:42.579187 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.579197 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.580329 (+ 1132us) server_negotiation.cc:407] Connection header received
0809 06:28:42.580469 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.580482 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.580752 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.580942 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:42.582100 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.583783 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.586632 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.587945 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.588097 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.588358 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.589623 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.589804 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":463,"threads_started":1}
I20260809 06:28:42.590469 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.579092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.8.131.2:37521)
0809 06:28:42.580393 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.580399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.580880 (+ 481us) server_negotiation.cc:407] Connection header received
0809 06:28:42.581095 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.581103 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.582211 (+ 1108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.583033 (+ 822us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:42.583206 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.584521 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.587359 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.588864 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.589020 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.589220 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.589999 (+ 779us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.590176 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":1044,"threads_started":1}
I20260809 06:28:42.590564 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.578423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:37521 (local address 127.0.0.1:58828)
0809 06:28:42.580756 (+ 2333us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:42.580774 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:42.580936 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:42.582454 (+ 1518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:42.582462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:42.582497 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:42.582850 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.582871 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.584694 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.584699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.587169 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.587184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.589647 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.589652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.589782 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.589804 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 06:28:42.589938 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 06:28:42.590068 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2239,"mutex_wait_us":87,"thread_start_us":2117,"threads_started":1}
I20260809 06:28:42.591804 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.580951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:42923 (local address 127.0.0.1:46990)
0809 06:28:42.581742 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:42.581770 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:42.581932 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:42.583508 (+ 1576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:42.583513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:42.583538 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:42.583952 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.583964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.585942 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.585947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.588493 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.588516 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.590980 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.590987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.591156 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.591187 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 06:28:42.591275 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 06:28:42.591404 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":1,"thread_start_us":495,"threads_started":1}
I20260809 06:28:42.591971 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.581548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46990 (local address 127.8.131.3:42923)
0809 06:28:42.582981 (+ 1433us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.582989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.583020 (+ 31us) server_negotiation.cc:407] Connection header received
0809 06:28:42.583103 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.583109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.583310 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.583489 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:42.584166 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.585793 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.589030 (+ 3237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.590476 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.590614 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.590817 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.591416 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.591587 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1253,"thread_start_us":1080,"threads_started":1}
I20260809 06:28:42.594327 30509 tablet_service.cc:1467] Processing CreateTablet for tablet 494c12f3f04f469bb2dff59bf722459a (DEFAULT_TABLE table=test-table [id=df192c0eec574f5297cae8e71141ff4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:28:42.594362 30700 tablet_service.cc:1467] Processing CreateTablet for tablet 494c12f3f04f469bb2dff59bf722459a (DEFAULT_TABLE table=test-table [id=df192c0eec574f5297cae8e71141ff4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:28:42.595633 30509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494c12f3f04f469bb2dff59bf722459a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:42.595798 30700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494c12f3f04f469bb2dff59bf722459a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:42.595458 30881 tablet_service.cc:1467] Processing CreateTablet for tablet 494c12f3f04f469bb2dff59bf722459a (DEFAULT_TABLE table=test-table [id=df192c0eec574f5297cae8e71141ff4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:28:42.596781 30881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494c12f3f04f469bb2dff59bf722459a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:28:42.602211 30607 heartbeater.cc:503] Master 127.8.131.62:39689 was elected leader, sending a full tablet report...
I20260809 06:28:42.608281 31006 tablet_bootstrap.cc:492] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Bootstrap starting.
I20260809 06:28:42.610927 31008 tablet_bootstrap.cc:492] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: Bootstrap starting.
I20260809 06:28:42.612569 31006 tablet_bootstrap.cc:654] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:42.614123 31006 log.cc:824] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:42.614666 31009 tablet_bootstrap.cc:492] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: Bootstrap starting.
I20260809 06:28:42.614698 31008 tablet_bootstrap.cc:654] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:42.616073 31008 log.cc:824] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:42.617805 31009 tablet_bootstrap.cc:654] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: Neither blocks nor log segments found. Creating new log.
I20260809 06:28:42.618660 31006 tablet_bootstrap.cc:492] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: No bootstrap required, opened a new log
I20260809 06:28:42.618943 31006 ts_tablet_manager.cc:1397] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260809 06:28:42.619083 31009 log.cc:824] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:42.620577 31008 tablet_bootstrap.cc:492] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: No bootstrap required, opened a new log
I20260809 06:28:42.620841 31008 ts_tablet_manager.cc:1397] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 06:28:42.622368 31009 tablet_bootstrap.cc:492] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: No bootstrap required, opened a new log
I20260809 06:28:42.622673 31009 ts_tablet_manager.cc:1397] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260809 06:28:42.627034 31006 raft_consensus.cc:348] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:42.627409 31006 raft_consensus.cc:374] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:42.627526 31006 raft_consensus.cc:729] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e987c4bda34cea887c2972d7916340, State: Initialized, Role: FOLLOWER
I20260809 06:28:42.628216 31006 consensus_queue.cc:260] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:42.630101 31006 ts_tablet_manager.cc:1428] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260809 06:28:42.631560 31008 raft_consensus.cc:348] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:42.632067 31008 raft_consensus.cc:374] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:42.632164 31008 raft_consensus.cc:729] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de08f03f2ea146e795ea71ddeb3e6df8, State: Initialized, Role: FOLLOWER
I20260809 06:28:42.632753 31008 consensus_queue.cc:260] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:42.633251 31009 raft_consensus.cc:348] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:42.633775 31009 raft_consensus.cc:374] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:28:42.633876 31009 raft_consensus.cc:729] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f8b23048bd4c5e8dfe2e580b0dd000, State: Initialized, Role: FOLLOWER
I20260809 06:28:42.634346 30790 heartbeater.cc:503] Master 127.8.131.62:39689 was elected leader, sending a full tablet report...
I20260809 06:28:42.634548 31009 consensus_queue.cc:260] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:42.635182 31008 ts_tablet_manager.cc:1428] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 06:28:42.636317 30978 heartbeater.cc:503] Master 127.8.131.62:39689 was elected leader, sending a full tablet report...
I20260809 06:28:42.638141 31009 ts_tablet_manager.cc:1428] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
W20260809 06:28:42.766348 30979 tablet.cc:2369] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:28:42.772858 30792 tablet.cc:2369] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:28:42.796497 31015 raft_consensus.cc:482] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:28:42.796798 31015 raft_consensus.cc:504] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:42.798404 31015 leader_election.cc:288] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97e987c4bda34cea887c2972d7916340 (127.8.131.1:45833), de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521)
I20260809 06:28:42.805045 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.799008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:41843 (local address 127.8.131.2:37521)
0809 06:28:42.799208 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.799214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.799239 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:28:42.799484 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.799492 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.799579 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.799726 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:42.800085 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.801100 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.803008 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.804205 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.804304 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.804372 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.804744 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.804828 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 06:28:42.805161 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.798704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:37521 (local address 127.8.131.3:41843)
0809 06:28:42.799084 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:42.799107 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:42.799272 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:42.799748 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:42.799751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:42.799785 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:42.799950 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.799958 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.801231 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.801237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.802845 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.802860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.804364 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.804370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.804571 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.804608 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 06:28:42.804718 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 06:28:42.804856 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":3}
I20260809 06:28:42.806042 30999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.799005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:37733 (local address 127.8.131.1:45833)
0809 06:28:42.799208 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.799214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.799937 (+ 723us) server_negotiation.cc:407] Connection header received
0809 06:28:42.800154 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.800159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.800220 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.800292 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:28:42.800918 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.801916 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.804024 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.805166 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.805302 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.805413 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.805762 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.805872 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 06:28:42.806205 31016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.798789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.8.131.3:37733)
0809 06:28:42.799805 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:42.799823 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:42.799963 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:42.800349 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:42.800356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:42.800382 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:28:42.800755 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.800767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.802079 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.802083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.803875 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:42.803891 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.805450 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.805456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:42.805596 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.805618 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 06:28:42.805694 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 06:28:42.805807 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":25,"thread_start_us":641,"threads_started":1}
I20260809 06:28:42.806340 30727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494c12f3f04f469bb2dff59bf722459a" candidate_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" is_pre_election: true
I20260809 06:28:42.806967 30727 raft_consensus.cc:2454] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23f8b23048bd4c5e8dfe2e580b0dd000 in term 0.
I20260809 06:28:42.807330 30539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494c12f3f04f469bb2dff59bf722459a" candidate_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e987c4bda34cea887c2972d7916340" is_pre_election: true
I20260809 06:28:42.807742 30820 leader_election.cc:302] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f8b23048bd4c5e8dfe2e580b0dd000, de08f03f2ea146e795ea71ddeb3e6df8; no voters:
I20260809 06:28:42.807845 30539 raft_consensus.cc:2454] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23f8b23048bd4c5e8dfe2e580b0dd000 in term 0.
I20260809 06:28:42.808199 31015 raft_consensus.cc:2781] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:28:42.808434 31015 raft_consensus.cc:482] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:28:42.808583 31015 raft_consensus.cc:3037] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:42.810864 31015 raft_consensus.cc:504] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:42.811425 31015 leader_election.cc:288] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [CANDIDATE]: Term 1 election: Requested vote from peers 97e987c4bda34cea887c2972d7916340 (127.8.131.1:45833), de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521)
I20260809 06:28:42.811940 30727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494c12f3f04f469bb2dff59bf722459a" candidate_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de08f03f2ea146e795ea71ddeb3e6df8"
I20260809 06:28:42.811940 30539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494c12f3f04f469bb2dff59bf722459a" candidate_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e987c4bda34cea887c2972d7916340"
I20260809 06:28:42.812140 30539 raft_consensus.cc:3037] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:42.812140 30727 raft_consensus.cc:3037] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:28:42.815349 30539 raft_consensus.cc:2454] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23f8b23048bd4c5e8dfe2e580b0dd000 in term 1.
I20260809 06:28:42.815349 30727 raft_consensus.cc:2454] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23f8b23048bd4c5e8dfe2e580b0dd000 in term 1.
I20260809 06:28:42.815788 30820 leader_election.cc:302] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f8b23048bd4c5e8dfe2e580b0dd000, de08f03f2ea146e795ea71ddeb3e6df8; no voters:
I20260809 06:28:42.816069 31015 raft_consensus.cc:2781] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:28:42.816476 31015 raft_consensus.cc:686] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 1 LEADER]: Becoming Leader. State: Replica: 23f8b23048bd4c5e8dfe2e580b0dd000, State: Running, Role: LEADER
I20260809 06:28:42.816936 31015 consensus_queue.cc:237] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
W20260809 06:28:42.817360 30609 tablet.cc:2369] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:28:42.821702 30347 catalog_manager.cc:5495] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3). New cstate: current_term: 1 leader_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } health_report { overall_health: HEALTHY } } }
I20260809 06:28:42.889350 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.884384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.8.131.3:42923)
0809 06:28:42.884566 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.884570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.884600 (+ 30us) server_negotiation.cc:407] Connection header received
0809 06:28:42.884768 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.884771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.884859 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.884985 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:28:42.885326 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.886040 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.887372 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.887836 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.887932 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.888395 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:28:42.888879 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:28:42.888993 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.889086 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.889151 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 06:28:42.897265 30727 raft_consensus.cc:1261] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [term 1 FOLLOWER]: Refusing update from remote peer 23f8b23048bd4c5e8dfe2e580b0dd000: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:28:42.897332 30539 raft_consensus.cc:1261] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Refusing update from remote peer 23f8b23048bd4c5e8dfe2e580b0dd000: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:28:42.898244 31018 consensus_queue.cc:1035] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:28:42.898617 31015 consensus_queue.cc:1035] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:28:42.913178 31024 mvcc.cc:205] Tried to move back new op lower bound from 7316508356177125376 to 7316508355862056960. Current Snapshot: MvccSnapshot[applied={T|T < 7316508356177125376}]
I20260809 06:28:42.925158 30348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:28:42.925432 30348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:28:42.929037 30348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Sending DeleteTablet for 3 replicas of tablet 494c12f3f04f469bb2dff59bf722459a
I20260809 06:28:42.930171 30509 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c12f3f04f469bb2dff59bf722459a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:28:42 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:42604
I20260809 06:28:42.930540 30700 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c12f3f04f469bb2dff59bf722459a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:28:42 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:58828
I20260809 06:28:42.930682 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 30398
I20260809 06:28:42.930892 30881 tablet_service.cc:1514] Processing DeleteTablet for tablet 494c12f3f04f469bb2dff59bf722459a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:28:42 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:46990
I20260809 06:28:42.948261 30999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:42.931675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42620 (local address 127.8.131.1:45833)
0809 06:28:42.932176 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:42.932180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:42.932193 (+ 13us) server_negotiation.cc:407] Connection header received
0809 06:28:42.932235 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:42.932238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:42.932292 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:42.932403 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:42.934298 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.935118 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.937127 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:42.937591 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:42.937736 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:42.941542 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:42.941554 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:42.941567 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:42.941599 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:42.944722 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:42.945831 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:42.945838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:42.945841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:42.946000 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:42.946546 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:42.946550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:42.946552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:42.946917 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:42.947237 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:42.947573 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 06:28:42.947836 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402}
I20260809 06:28:42.949007 30599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:42.956295 31041 tablet_replica.cc:330] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: stopping tablet replica
I20260809 06:28:42.956945 31041 raft_consensus.cc:2229] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:28:42.957836 31041 raft_consensus.cc:2258] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:28:42.960587 31041 ts_tablet_manager.cc:1905] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:28:42.961201 31040 tablet_replica.cc:330] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: stopping tablet replica
I20260809 06:28:42.961643 31040 raft_consensus.cc:2229] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:28:42.963382 31040 raft_consensus.cc:2258] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:28:42.966392 31040 ts_tablet_manager.cc:1905] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:28:42.968894 31041 ts_tablet_manager.cc:1918] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:28:42.969072 31041 log.cc:1196] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/494c12f3f04f469bb2dff59bf722459a
I20260809 06:28:42.969728 31041 ts_tablet_manager.cc:1939] T 494c12f3f04f469bb2dff59bf722459a P 23f8b23048bd4c5e8dfe2e580b0dd000: Deleting consensus metadata
I20260809 06:28:42.971791 30327 catalog_manager.cc:4841] TS 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923): tablet 494c12f3f04f469bb2dff59bf722459a (table test-table [id=df192c0eec574f5297cae8e71141ff4c]) successfully deleted
I20260809 06:28:42.974157 31040 ts_tablet_manager.cc:1918] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:28:42.974393 31040 log.cc:1196] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/494c12f3f04f469bb2dff59bf722459a
I20260809 06:28:42.975031 31040 ts_tablet_manager.cc:1939] T 494c12f3f04f469bb2dff59bf722459a P de08f03f2ea146e795ea71ddeb3e6df8: Deleting consensus metadata
I20260809 06:28:42.976946 30327 catalog_manager.cc:4841] TS de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521): tablet 494c12f3f04f469bb2dff59bf722459a (table test-table [id=df192c0eec574f5297cae8e71141ff4c]) successfully deleted
I20260809 06:28:43.064901 31039 tablet_replica.cc:330] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: stopping tablet replica
I20260809 06:28:43.065231 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 30398
W20260809 06:28:43.094264 30326 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.131.1:45833 (error 108)
I20260809 06:28:43.094827 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 30612
I20260809 06:28:43.114786 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.096534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.8.131.2:37521)
0809 06:28:43.096718 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:43.096724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:43.096742 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:28:43.096810 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:43.096818 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:43.096907 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:43.097006 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:43.098950 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:43.100176 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:43.102119 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:43.102791 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:43.102903 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:43.106726 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:43.106745 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:43.106765 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:43.106806 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:43.110751 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:43.111654 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:43.111664 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:43.111667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:43.111811 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:43.112253 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:43.112258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:43.112262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:43.112747 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:43.113024 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:43.114169 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0809 06:28:43.114383 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 06:28:43.115809 30782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:43.116707 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.110138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42624)
0809 06:28:43.116163 (+ 6025us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:43.116330 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5850,"thread_start_us":5672,"threads_started":1}
W20260809 06:28:43.117154 30326 catalog_manager.cc:4584] TS 97e987c4bda34cea887c2972d7916340 (127.8.131.1:45833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 494c12f3f04f469bb2dff59bf722459a: Network error: Client connection negotiation failed: client connection to 127.8.131.1:45833: connect: Connection refused (error 111)
I20260809 06:28:43.171255 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.170683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42634)
0809 06:28:43.170953 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:43.171082 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 06:28:43.172435 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.171992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42650)
0809 06:28:43.172179 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:43.172262 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 06:28:43.222587 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 30612
I20260809 06:28:43.233283 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.232670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42664)
0809 06:28:43.232936 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:43.233066 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 06:28:43.234653 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.234177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42668)
0809 06:28:43.234365 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:43.234472 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 06:28:43.247633 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 30795
I20260809 06:28:43.264096 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.248479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47016 (local address 127.8.131.3:42923)
0809 06:28:43.248686 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:43.248692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:43.248731 (+ 39us) server_negotiation.cc:407] Connection header received
0809 06:28:43.248903 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:43.248909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:43.249002 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:43.249170 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:43.250549 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:43.251785 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:43.253112 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:43.253871 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:43.254016 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:43.256938 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:43.256952 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:43.256973 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:43.257019 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:43.261157 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:43.261804 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:43.261815 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:43.261818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:43.262037 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:43.262421 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:43.262427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:43.262430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:43.262889 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:43.263214 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:43.263404 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 06:28:43.263623 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:28:43.264938 30970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:43.315155 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.314611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42684)
0809 06:28:43.314847 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:43.314956 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 06:28:43.316344 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.315883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42692)
0809 06:28:43.316097 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:43.316181 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 06:28:43.390079 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 30795
I20260809 06:28:43.419301 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 30307
I20260809 06:28:43.420595 30388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:28:43.534408 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 30307
I20260809 06:28:43.534500 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:43.533731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45833 (local address 127.0.0.1:42696)
0809 06:28:43.534194 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:43.534343 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274}
I20260809 06:28:43.564246 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:39689
--webserver_interface=127.8.131.62
--webserver_port=38655
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:43.915272 31049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:43.915592 31049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:43.915781 31049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:43.923563 31049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:28:43.923723 31049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:43.923799 31049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:43.923853 31049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:28:43.923899 31049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:28:43.933979 31049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:39689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=38655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:43.935271 31049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:43.936573 31049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:43.949028 31054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:43.949266 31060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:43.949330 31055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:43.950294 31049 server_base.cc:1034] running on GCE node
I20260809 06:28:43.951162 31049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:43.952705 31049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:43.953956 31049 hybrid_clock.cc:648] HybridClock initialized: now 1786256923953911 us; error 56 us; skew 500 ppm
I20260809 06:28:43.954319 31049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:43 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256923, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:43.957568 31049 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:28:43.959218 31049 webserver.cc:460] Webserver started at http://127.8.131.62:38655/ using document root <none> and password file <none>
I20260809 06:28:43.960008 31049 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:43.960163 31049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:43.965543 31049 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:28:43.968371 31070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:43.969667 31049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:28:43.969863 31049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "11c0d60f3f084aa0a150d6466a299bcb"
format_stamp: "Formatted at 2026-08-09 06:28:37 on dist-test-slave-mjjr"
server_key: "599389be7e885c300a3fa81b84c38503"
server_key_iv: "6f3d7b267b0b8d4b87b4a3b2e38e57ff"
server_key_version: "encryptionkey@0"
I20260809 06:28:43.970862 31049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:43.991740 31049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:43.995491 31049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:43.995783 31049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:44.014907 31049 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:39689
I20260809 06:28:44.014937 31145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:39689 every 8 connection(s)
I20260809 06:28:44.016513 31049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:28:44.019912 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 31049
I20260809 06:28:44.020893 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:45833
--local_ip_for_outbound_sockets=127.8.131.1
--tserver_master_addrs=127.8.131.62:39689
--webserver_port=36799
--webserver_interface=127.8.131.1
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:28:44.023964 31146 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:28:44.027158 31146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Bootstrap starting.
I20260809 06:28:44.055603 31146 log.cc:824] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:44.077068 31146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:28:44.077770 31146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb: Bootstrap complete.
I20260809 06:28:44.092121 31146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:44.093678 31146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c0d60f3f084aa0a150d6466a299bcb, State: Initialized, Role: FOLLOWER
I20260809 06:28:44.094487 31146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:44.094843 31146 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:28:44.095000 31146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:28:44.095228 31146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 2 FOLLOWER]: Advancing to term 3
I20260809 06:28:44.101544 31146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:44.102197 31146 leader_election.cc:302] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11c0d60f3f084aa0a150d6466a299bcb; no voters:
I20260809 06:28:44.102895 31146 leader_election.cc:288] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 06:28:44.103111 31150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 06:28:44.104578 31150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [term 3 LEADER]: Becoming Leader. State: Replica: 11c0d60f3f084aa0a150d6466a299bcb, State: Running, Role: LEADER
I20260809 06:28:44.105127 31146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:28:44.105147 31150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } }
I20260809 06:28:44.107822 31152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "11c0d60f3f084aa0a150d6466a299bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } } }
I20260809 06:28:44.108182 31152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:44.108724 31153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11c0d60f3f084aa0a150d6466a299bcb. Latest consensus state: current_term: 3 leader_uuid: "11c0d60f3f084aa0a150d6466a299bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c0d60f3f084aa0a150d6466a299bcb" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 39689 } } }
I20260809 06:28:44.109041 31153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c0d60f3f084aa0a150d6466a299bcb [sys.catalog]: This master's current role is: LEADER
I20260809 06:28:44.116573 31160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:28:44.126137 31160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:28:44.127924 31160 catalog_manager.cc:1259] Loaded cluster ID: d90f199925ae494d9608e00c4dc8b496
I20260809 06:28:44.128072 31160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 06:28:44.130144 31160 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f935ea6f9ad google::LogMessage::Fail() at ??:0
@ 0x7f935ea7288a google::LogMessage::SendToLog() at ??:0
@ 0x7f935ea6f4de google::LogMessage::Flush() at ??:0
@ 0x7f935ea70a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f93744a2512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f937449f976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9374527610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f9374527439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f93616bb345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f93616be91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f93616be799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9361699eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f93649736db start_thread at ??:0
@ 0x7f935d90961f clone at ??:0
W20260809 06:28:44.467404 31148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:44.467693 31148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:44.467794 31148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:44.474771 31148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:44.474879 31148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:44.474996 31148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:28:44.484550 31148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:45833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=36799
--tserver_master_addrs=127.8.131.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:44.485759 31148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:44.487367 31148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:44.501071 31209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:44.501158 31203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:44.501178 31205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:44.501991 31148 server_base.cc:1034] running on GCE node
I20260809 06:28:44.502635 31148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:44.503677 31148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:44.504927 31148 hybrid_clock.cc:648] HybridClock initialized: now 1786256924504863 us; error 54 us; skew 500 ppm
I20260809 06:28:44.505254 31148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:44 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256924, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:44.508620 31148 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:28:44.510079 31148 webserver.cc:460] Webserver started at http://127.8.131.1:36799/ using document root <none> and password file <none>
I20260809 06:28:44.510795 31148 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:44.510948 31148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:44.516214 31148 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:28:44.519105 31216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:44.520428 31148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:28:44.520601 31148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "97e987c4bda34cea887c2972d7916340"
format_stamp: "Formatted at 2026-08-09 06:28:38 on dist-test-slave-mjjr"
server_key: "f431ac96fdbb619008388a43b687c944"
server_key_iv: "c5ce194394ad76e853ef8a15bf4f112a"
server_key_version: "encryptionkey@0"
I20260809 06:28:44.521638 31148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:44.542626 31148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:44.547088 31148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:44.547377 31148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:44.548419 31148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:44.550242 31227 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 06:28:44.555812 31148 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 06:28:44.555939 31148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 06:28:44.556084 31148 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 06:28:44.559106 31148 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 06:28:44.559219 31148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260809 06:28:44.559514 31227 tablet_bootstrap.cc:492] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Bootstrap starting.
I20260809 06:28:44.587276 31227 log.cc:824] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Log is configured to *not* fsync() on all Append() calls
I20260809 06:28:44.601589 31227 tablet_bootstrap.cc:492] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:28:44.602164 31227 tablet_bootstrap.cc:492] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Bootstrap complete.
I20260809 06:28:44.603780 31148 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:45833
I20260809 06:28:44.603790 31381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:45833 every 8 connection(s)
I20260809 06:28:44.603837 31227 ts_tablet_manager.cc:1397] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Time spent bootstrapping tablet: real 0.045s user 0.032s sys 0.011s
I20260809 06:28:44.605067 31148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:28:44.605912 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 31148
I20260809 06:28:44.606757 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:37521
--local_ip_for_outbound_sockets=127.8.131.2
--tserver_master_addrs=127.8.131.62:39689
--webserver_port=35857
--webserver_interface=127.8.131.2
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:28:44.614678 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:44.612620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:42611)
0809 06:28:44.613512 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:44.613778 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":382,"threads_started":1}
W20260809 06:28:44.615455 31382 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:39689 (0 consecutive failures): Network error: Failed to ping master at 127.8.131.62:39689: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
I20260809 06:28:44.614985 31227 raft_consensus.cc:348] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:44.616384 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:44.616033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:60501)
0809 06:28:44.616190 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:44.616276 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 06:28:44.616501 31227 raft_consensus.cc:729] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e987c4bda34cea887c2972d7916340, State: Initialized, Role: FOLLOWER
I20260809 06:28:44.617177 31227 consensus_queue.cc:260] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:44.617540 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:44.617116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:50629)
0809 06:28:44.617280 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:44.617368 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 06:28:44.617784 31382 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 06:28:44.618956 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:44.618614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:52387)
0809 06:28:44.618756 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:44.618834 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 06:28:44.619167 31227 ts_tablet_manager.cc:1428] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260809 06:28:44.622073 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:44.621757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:60293)
0809 06:28:44.621895 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:44.621968 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 06:28:44.961975 31386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:44.962319 31386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:44.962499 31386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:44.969689 31386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:44.969846 31386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:44.970036 31386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:28:44.979811 31386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:37521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=35857
--tserver_master_addrs=127.8.131.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:44.981253 31386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:44.982626 31386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:44.996248 31395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:44.996120 31398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:44.996170 31396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:44.996938 31386 server_base.cc:1034] running on GCE node
I20260809 06:28:44.997535 31386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:44.998474 31386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:44.999758 31386 hybrid_clock.cc:648] HybridClock initialized: now 1786256924999698 us; error 55 us; skew 500 ppm
I20260809 06:28:45.000090 31386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:44 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256924, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:45.003226 31386 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:28:45.004705 31386 webserver.cc:460] Webserver started at http://127.8.131.2:35857/ using document root <none> and password file <none>
I20260809 06:28:45.005365 31386 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:45.005470 31386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:45.011173 31386 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 06:28:45.013954 31409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:45.015282 31386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:28:45.015429 31386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "de08f03f2ea146e795ea71ddeb3e6df8"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "f75bb010ba71ea3710777181c7067715"
server_key_iv: "66637460cb0a3b08e32975b27e430500"
server_key_version: "encryptionkey@0"
I20260809 06:28:45.016458 31386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:45.033991 31386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:45.037878 31386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:45.038110 31386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:45.039024 31386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:45.040359 31386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:45.040452 31386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:45.040531 31386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:45.040591 31386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:45.074263 31386 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:37521
I20260809 06:28:45.074353 31568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:37521 every 8 connection(s)
I20260809 06:28:45.075763 31386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:28:45.080405 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:45.078568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:45073)
0809 06:28:45.079300 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:45.079510 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":282,"threads_started":1}
W20260809 06:28:45.081105 31569 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:39689 (0 consecutive failures): Network error: Failed to ping master at 127.8.131.62:39689: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
I20260809 06:28:45.081974 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:45.081588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:38853)
0809 06:28:45.081776 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:45.081855 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 06:28:45.082999 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:45.082552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:58605)
0809 06:28:45.082739 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:45.082841 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 06:28:45.083065 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 31386
W20260809 06:28:45.083294 31569 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 06:28:45.083843 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:42923
--local_ip_for_outbound_sockets=127.8.131.3
--tserver_master_addrs=127.8.131.62:39689
--webserver_port=36363
--webserver_interface=127.8.131.3
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:28:45.423542 31572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:28:45.423887 31572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:28:45.424083 31572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:28:45.431682 31572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:28:45.431818 31572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:28:45.432004 31572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:28:45.441751 31572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:42923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=36363
--tserver_master_addrs=127.8.131.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:28:45.443048 31572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:28:45.444448 31572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:28:45.457616 31589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:45.457583 31582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:28:45.457659 31583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:28:45.458818 31572 server_base.cc:1034] running on GCE node
I20260809 06:28:45.459502 31572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:28:45.460623 31572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:28:45.461839 31572 hybrid_clock.cc:648] HybridClock initialized: now 1786256925461797 us; error 30 us; skew 500 ppm
I20260809 06:28:45.462193 31572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:28:45 dist-test-slave-mjjr krb5kdc[29522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256925, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:28:45.465477 31572 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:28:45.467051 31572 webserver.cc:460] Webserver started at http://127.8.131.3:36363/ using document root <none> and password file <none>
I20260809 06:28:45.467892 31572 fs_manager.cc:359] Metadata directory not provided
I20260809 06:28:45.468042 31572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:28:45.473519 31572 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 06:28:45.476271 31596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:45.477641 31572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:28:45.477828 31572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "23f8b23048bd4c5e8dfe2e580b0dd000"
format_stamp: "Formatted at 2026-08-09 06:28:39 on dist-test-slave-mjjr"
server_key: "2baa576310022fce57dab0eaa1c10af4"
server_key_iv: "d74194a1070a35810d7baf90e3754784"
server_key_version: "encryptionkey@0"
I20260809 06:28:45.478852 31572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:28:45.499347 31572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:28:45.503408 31572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:28:45.503721 31572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:28:45.504727 31572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:28:45.506062 31572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:28:45.506187 31572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:45.506340 31572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:28:45.506430 31572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:28:45.541738 31572 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:42923
I20260809 06:28:45.541770 31765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:42923 every 8 connection(s)
I20260809 06:28:45.543187 31572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:28:45.547996 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:45.546136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:51021)
0809 06:28:45.546867 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:45.547072 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":307,"threads_started":1}
W20260809 06:28:45.548641 31767 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:39689 (0 consecutive failures): Network error: Failed to ping master at 127.8.131.62:39689: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
I20260809 06:28:45.549376 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:45.549072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:60245)
0809 06:28:45.549209 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:45.549273 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 06:28:45.550151 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:45.549872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:50343)
0809 06:28:45.549987 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:45.550055 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 06:28:45.550500 31767 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 06:28:45.552086 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 31572
I20260809 06:28:45.623890 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:45.622837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:40209)
0809 06:28:45.623414 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:45.623552 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":273,"threads_started":1}
I20260809 06:28:46.084694 31773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:46.083886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:45971)
0809 06:28:46.084354 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:46.084462 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":226,"threads_started":1}
I20260809 06:28:46.315979 31774 raft_consensus.cc:482] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:28:46.316282 31774 raft_consensus.cc:504] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:46.317937 31774 leader_election.cc:288] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521), 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923)
Aug 09 06:28:46 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256924, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
Aug 09 06:28:46 dist-test-slave-mjjr krb5kdc[29522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256924, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:28:46.345235 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:46.318749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:42923 (local address 127.8.131.1:32881)
0809 06:28:46.320120 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:46.320134 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:46.320234 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:46.323202 (+ 2968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:46.323206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:46.323756 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:46.324325 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:46.324352 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:46.326018 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:46.326024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:46.328075 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:46.328088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:46.329055 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:46.329061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:46.329402 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:46.334775 (+ 5373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:46.334798 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:46.337504 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:46.341387 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:46.341396 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:46.341418 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:46.341964 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:46.342440 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:46.342445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:46.342448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:46.342683 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:46.343487 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:46.343495 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:46.343924 (+ 429us) client_negotiation.cc:769] Sending connection context
0809 06:28:46.344191 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 06:28:46.344735 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1231,"mutex_wait_us":196,"spinlock_wait_cycles":33664,"thread_start_us":1043,"threads_started":1}
I20260809 06:28:46.345253 31780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:46.319039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:32881 (local address 127.8.131.3:42923)
0809 06:28:46.320855 (+ 1816us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:46.320865 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:46.320903 (+ 38us) server_negotiation.cc:407] Connection header received
0809 06:28:46.322441 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:46.322452 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:46.322873 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:46.323225 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:46.324536 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:46.325820 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:46.328214 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:46.328826 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:46.329043 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:46.337939 (+ 8896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:46.337953 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:46.337979 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:46.338040 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:46.341224 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:46.342111 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:46.342131 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:46.342137 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:46.342300 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:46.342809 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:46.342820 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:46.342823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:46.343334 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:46.343553 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:46.344434 (+ 881us) server_negotiation.cc:299] Negotiation successful
0809 06:28:46.344828 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1674,"thread_start_us":1498,"threads_started":1}
I20260809 06:28:46.345263 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:46.318503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:36155 (local address 127.8.131.2:37521)
0809 06:28:46.319457 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0809 06:28:46.319478 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 06:28:46.319586 (+ 108us) server_negotiation.cc:407] Connection header received
0809 06:28:46.321729 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:28:46.321738 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:28:46.322097 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:28:46.322359 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:28:46.324566 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:46.325842 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:46.328215 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:28:46.328833 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:28:46.329049 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:46.338318 (+ 9269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:28:46.338326 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:28:46.338336 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:28:46.338374 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:28:46.341265 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:46.342128 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:46.342134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:46.342140 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:46.342306 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:28:46.342804 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:28:46.342809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:28:46.342812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:28:46.343335 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:28:46.343553 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 06:28:46.344458 (+ 905us) server_negotiation.cc:299] Negotiation successful
0809 06:28:46.344841 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":678,"threads_started":1}
I20260809 06:28:46.345355 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:46.318113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:37521 (local address 127.8.131.1:36155)
0809 06:28:46.318671 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:46.318715 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 06:28:46.319357 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:28:46.322574 (+ 3217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:28:46.322585 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:28:46.323595 (+ 1010us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:28:46.324326 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:46.324351 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:46.326082 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:46.326088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:46.328076 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:28:46.328088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:28:46.329135 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:28:46.329143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:28:46.329402 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:28:46.334883 (+ 5481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:28:46.334900 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:28:46.337899 (+ 2999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:28:46.341413 (+ 3514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:46.341421 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:46.341427 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:46.341977 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:46.342443 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:28:46.342449 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:28:46.342453 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:28:46.342684 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:28:46.343487 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:28:46.343495 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:28:46.343923 (+ 428us) client_negotiation.cc:769] Sending connection context
0809 06:28:46.344191 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 06:28:46.344895 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":438,"spinlock_wait_cycles":1408,"thread_start_us":265,"threads_started":1}
W20260809 06:28:46.349241 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
W20260809 06:28:46.349576 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
I20260809 06:28:46.349717 31220 leader_election.cc:302] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97e987c4bda34cea887c2972d7916340; no voters: 23f8b23048bd4c5e8dfe2e580b0dd000, de08f03f2ea146e795ea71ddeb3e6df8
I20260809 06:28:46.350129 31774 raft_consensus.cc:2726] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 06:28:46.551883 31781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:46.551003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:52153)
0809 06:28:46.551451 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:46.551587 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":209,"threads_started":1}
I20260809 06:28:46.625202 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:46.624771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:48185)
0809 06:28:46.624958 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:46.625062 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 06:28:47.086292 31782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:47.085631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:33139)
0809 06:28:47.086054 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:47.086133 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":241,"threads_started":1}
I20260809 06:28:47.553244 31783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:47.552620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:48253)
0809 06:28:47.553011 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:47.553104 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":206,"threads_started":1}
I20260809 06:28:47.626878 31786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:47.626084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:45435)
0809 06:28:47.626615 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:47.626701 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":216,"threads_started":1}
I20260809 06:28:48.087878 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:48.087120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:58435)
0809 06:28:48.087567 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:48.087704 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":228,"threads_started":1}
I20260809 06:28:48.447387 31790 raft_consensus.cc:482] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:28:48.447618 31790 raft_consensus.cc:504] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:48.448331 31790 leader_election.cc:288] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521), 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923)
W20260809 06:28:48.449378 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
W20260809 06:28:48.449652 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
I20260809 06:28:48.449764 31220 leader_election.cc:302] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97e987c4bda34cea887c2972d7916340; no voters: 23f8b23048bd4c5e8dfe2e580b0dd000, de08f03f2ea146e795ea71ddeb3e6df8
I20260809 06:28:48.450062 31790 raft_consensus.cc:2726] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 06:28:48.554870 31791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:48.554100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:55627)
0809 06:28:48.554635 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:48.554713 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":327,"threads_started":1}
I20260809 06:28:48.628700 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:48.627914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:36883)
0809 06:28:48.628380 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:48.628487 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":218,"threads_started":1}
I20260809 06:28:49.089586 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:49.088740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:43277)
0809 06:28:49.089274 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:49.089356 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":316,"threads_started":1}
I20260809 06:28:49.556469 31795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:49.555791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:38941)
0809 06:28:49.556212 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:49.556302 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":211,"threads_started":1}
I20260809 06:28:49.630403 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:49.629710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:44417)
0809 06:28:49.630159 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:49.630242 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":257,"threads_started":1}
I20260809 06:28:50.035328 31797 raft_consensus.cc:482] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:28:50.035548 31797 raft_consensus.cc:504] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:50.036216 31797 leader_election.cc:288] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521), 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923)
W20260809 06:28:50.037256 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
W20260809 06:28:50.037511 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
I20260809 06:28:50.037643 31220 leader_election.cc:302] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97e987c4bda34cea887c2972d7916340; no voters: 23f8b23048bd4c5e8dfe2e580b0dd000, de08f03f2ea146e795ea71ddeb3e6df8
I20260809 06:28:50.037921 31797 raft_consensus.cc:2726] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 06:28:50.091213 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:50.090528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:47375)
0809 06:28:50.090982 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:50.091060 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":229,"threads_started":1}
I20260809 06:28:50.558254 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:50.557426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:40775)
0809 06:28:50.558023 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:50.558101 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":316,"threads_started":1}
I20260809 06:28:50.632017 31801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:50.631230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:43629)
0809 06:28:50.631762 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:50.631854 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":281,"threads_started":1}
I20260809 06:28:51.092926 31802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:51.092210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:48819)
0809 06:28:51.092666 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:51.092744 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":257,"threads_started":1}
I20260809 06:28:51.560081 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:51.559132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:55647)
0809 06:28:51.559833 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:51.559919 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":442,"threads_started":1}
I20260809 06:28:51.633697 31805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:51.632831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:50153)
0809 06:28:51.633366 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:51.633476 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":338,"threads_started":1}
I20260809 06:28:52.094769 31806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:52.093951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:40041)
0809 06:28:52.094481 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:52.094585 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":232,"threads_started":1}
I20260809 06:28:52.561749 31807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:52.560993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:44191)
0809 06:28:52.561487 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:52.561570 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":232,"threads_started":1}
I20260809 06:28:52.635432 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:52.634694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:55113)
0809 06:28:52.635159 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:52.635256 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":247,"threads_started":1}
I20260809 06:28:53.096457 31809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:53.095726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:42319)
0809 06:28:53.096188 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:53.096277 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":248,"threads_started":1}
I20260809 06:28:53.563524 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:53.562703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:55555)
0809 06:28:53.563188 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:53.563268 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":241,"threads_started":1}
I20260809 06:28:53.637132 31811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:53.636407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:48141)
0809 06:28:53.636855 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:53.636931 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":223,"threads_started":1}
I20260809 06:28:54.098174 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:54.097409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:44119)
0809 06:28:54.097869 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:54.097982 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":224,"threads_started":1}
I20260809 06:28:54.565099 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:54.564390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:56631)
0809 06:28:54.564858 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:54.564944 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":228,"threads_started":1}
I20260809 06:28:54.638849 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:54.638157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:46963)
0809 06:28:54.638597 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:54.638683 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":237,"threads_started":1}
I20260809 06:28:54.952374 31820 raft_consensus.cc:482] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:28:54.952639 31820 raft_consensus.cc:504] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:54.953306 31820 leader_election.cc:288] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521), 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923)
W20260809 06:28:54.954358 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
W20260809 06:28:54.954578 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
I20260809 06:28:54.954655 31220 leader_election.cc:302] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97e987c4bda34cea887c2972d7916340; no voters: 23f8b23048bd4c5e8dfe2e580b0dd000, de08f03f2ea146e795ea71ddeb3e6df8
I20260809 06:28:54.954897 31820 raft_consensus.cc:2726] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 06:28:55.100090 31822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:55.099301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:35745)
0809 06:28:55.099813 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:55.099895 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":258,"threads_started":1}
I20260809 06:28:55.566715 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:55.566003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:45695)
0809 06:28:55.566434 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:55.566529 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":236,"threads_started":1}
I20260809 06:28:55.640612 31824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:55.639908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:42717)
0809 06:28:55.640355 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:55.640439 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":231,"threads_started":1}
I20260809 06:28:56.101807 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:56.101018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:45463)
0809 06:28:56.101528 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:56.101613 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":257,"threads_started":1}
I20260809 06:28:56.568521 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:56.567739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:34821)
0809 06:28:56.568283 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:56.568363 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":323,"threads_started":1}
I20260809 06:28:56.642336 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:56.641495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:50977)
0809 06:28:56.642022 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:56.642119 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":297,"threads_started":1}
I20260809 06:28:57.103528 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:57.102780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:34553)
0809 06:28:57.103269 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:57.103346 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":272,"threads_started":1}
I20260809 06:28:57.108361 31830 raft_consensus.cc:482] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:28:57.108560 31830 raft_consensus.cc:504] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e987c4bda34cea887c2972d7916340" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45833 } } peers { permanent_uuid: "de08f03f2ea146e795ea71ddeb3e6df8" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 37521 } } peers { permanent_uuid: "23f8b23048bd4c5e8dfe2e580b0dd000" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 42923 } }
I20260809 06:28:57.109272 31830 leader_election.cc:288] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521), 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923)
W20260809 06:28:57.110201 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de08f03f2ea146e795ea71ddeb3e6df8 (127.8.131.2:37521): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
W20260809 06:28:57.110492 31220 leader_election.cc:341] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f8b23048bd4c5e8dfe2e580b0dd000 (127.8.131.3:42923): Not found: Tablet not found: 494c12f3f04f469bb2dff59bf722459a
I20260809 06:28:57.110606 31220 leader_election.cc:302] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97e987c4bda34cea887c2972d7916340; no voters: 23f8b23048bd4c5e8dfe2e580b0dd000, de08f03f2ea146e795ea71ddeb3e6df8
I20260809 06:28:57.110886 31830 raft_consensus.cc:2726] T 494c12f3f04f469bb2dff59bf722459a P 97e987c4bda34cea887c2972d7916340 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 06:28:57.570178 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:57.569477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:33199)
0809 06:28:57.569939 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:57.570016 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":247,"threads_started":1}
I20260809 06:28:57.644150 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:57.643345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:37463)
0809 06:28:57.643863 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:57.643958 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":251,"threads_started":1}
I20260809 06:28:58.105307 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:58.104540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:53877)
0809 06:28:58.105042 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:58.105138 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":243,"threads_started":1}
I20260809 06:28:58.571940 31835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:58.571080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:60035)
0809 06:28:58.571616 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:58.571752 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":304,"threads_started":1}
I20260809 06:28:58.646032 31836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:58.645227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:59115)
0809 06:28:58.645760 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:58.645849 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":240,"threads_started":1}
I20260809 06:28:59.107299 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:59.106447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:53369)
0809 06:28:59.107005 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:59.107098 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":264,"threads_started":1}
I20260809 06:28:59.573863 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:59.572990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:49353)
0809 06:28:59.573541 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:59.573643 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":254,"threads_started":1}
I20260809 06:28:59.648073 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:28:59.647302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:50925)
0809 06:28:59.647820 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 06:28:59.647903 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":245,"threads_started":1}
I20260809 06:29:00.109146 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:00.108361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.2:48627)
0809 06:29:00.108855 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:00.108987 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":257,"threads_started":1}
I20260809 06:29:00.556054 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 31148
I20260809 06:29:00.571245 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:00.557256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59512 (local address 127.8.131.1:45833)
0809 06:29:00.557873 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:00.557883 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:00.557917 (+ 34us) server_negotiation.cc:407] Connection header received
0809 06:29:00.557997 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:00.558007 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:00.558242 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:00.558434 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:00.560167 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:00.561133 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:00.562381 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:00.562971 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:00.563116 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:00.565746 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:00.565757 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:00.565771 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:00.565803 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:00.568511 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:00.569178 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:00.569187 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:00.569190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:00.569323 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:00.569714 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:00.569719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:00.569722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:00.570054 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:00.570385 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:00.570671 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 06:29:00.570935 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":498,"threads_started":1}
I20260809 06:29:00.572515 31374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:00.576105 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:00.575147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.3:40291)
0809 06:29:00.575719 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:00.575890 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":262,"threads_started":1}
I20260809 06:29:00.661190 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:00.660012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:39689 (local address 127.8.131.1:49489)
0809 06:29:00.660614 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:00.660786 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":306,"threads_started":1}
I20260809 06:29:00.661279 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 31148
I20260809 06:29:00.688158 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 31386
I20260809 06:29:00.701354 31850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:00.688937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.8.131.2:37521)
0809 06:29:00.689469 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:00.689474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:00.689491 (+ 17us) server_negotiation.cc:407] Connection header received
0809 06:29:00.689549 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:00.689553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:00.689614 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:00.689709 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:00.691021 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:00.691921 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:00.693169 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:00.693589 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:00.693680 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:00.696327 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:00.696331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:00.696335 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:00.696367 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:00.699001 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:00.699685 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:00.699693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:00.699701 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:00.699825 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:00.700186 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:00.700189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:00.700191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:00.700459 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:00.700665 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:00.700887 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 06:29:00.701053 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":305,"threads_started":1}
I20260809 06:29:00.702112 31563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:00.789738 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 31386
I20260809 06:29:00.813202 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 31572
I20260809 06:29:00.826843 31852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:00.814088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33166 (local address 127.8.131.3:42923)
0809 06:29:00.814706 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:00.814710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:00.814729 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:29:00.814784 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:00.814788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:00.814859 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:00.814971 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:00.816439 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:00.817206 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:00.818427 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:00.818846 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:00.818938 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:00.821795 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:00.821800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:00.821803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:00.821834 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:00.824481 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:00.825150 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:00.825153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:00.825155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:00.825301 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:00.825609 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:00.825612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:00.825614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:00.825847 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:00.826053 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:00.826259 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 06:29:00.826427 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":348,"threads_started":1}
I20260809 06:29:00.827570 31759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:00.911198 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 31572
2026-08-09T06:29:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24939 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:29:01 dist-test-slave-mjjr krb5kdc[31859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:29:01 dist-test-slave-mjjr krb5kdc[31859](info): set up 2 sockets
Aug 09 06:29:01 dist-test-slave-mjjr krb5kdc[31859](info): commencing operation
krb5kdc: starting...
W20260809 06:29:02.232728 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:29:02 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:29:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:29:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:29:02.510624 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38899
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:38899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:29:02.853771 31875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:02.854128 31875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:02.854297 31875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:02.861501 31875 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 06:29:02.861630 31875 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 06:29:02.861720 31875 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 06:29:02.861821 31875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:29:02.861966 31875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:02.862073 31875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:02.862121 31875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:29:02.862177 31875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:29:02.872116 31875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:38899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:02.873337 31875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:02.874598 31875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:02.886889 31883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:02.887144 31875 server_base.cc:1034] running on GCE node
W20260809 06:29:02.886992 31886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:02.886935 31884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:02.888339 31875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:02.889775 31875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:02.891032 31875 hybrid_clock.cc:648] HybridClock initialized: now 1786256942890967 us; error 42 us; skew 500 ppm
I20260809 06:29:02.891381 31875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:02 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256942, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:02.894558 31875 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:29:02.896077 31875 webserver.cc:460] Webserver started at http://127.8.131.62:34311/ using document root <none> and password file <none>
I20260809 06:29:02.896775 31875 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:02.896924 31875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:02.897243 31875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:29:02.900295 31875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "318591c3c83e4609a9c2c9ab2a600dc5"
format_stamp: "Formatted at 2026-08-09 06:29:02 on dist-test-slave-mjjr"
server_key: "ec2cf10be60df12f0c50b508aea35f96"
server_key_iv: "49d97546cdb01ffb800d13fbf5b903df"
server_key_version: "encryptionkey@0"
I20260809 06:29:02.901111 31875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "318591c3c83e4609a9c2c9ab2a600dc5"
format_stamp: "Formatted at 2026-08-09 06:29:02 on dist-test-slave-mjjr"
server_key: "ec2cf10be60df12f0c50b508aea35f96"
server_key_iv: "49d97546cdb01ffb800d13fbf5b903df"
server_key_version: "encryptionkey@0"
I20260809 06:29:02.906347 31875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:29:02.909971 31895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:02.911335 31875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:29:02.911530 31875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "318591c3c83e4609a9c2c9ab2a600dc5"
format_stamp: "Formatted at 2026-08-09 06:29:02 on dist-test-slave-mjjr"
server_key: "ec2cf10be60df12f0c50b508aea35f96"
server_key_iv: "49d97546cdb01ffb800d13fbf5b903df"
server_key_version: "encryptionkey@0"
I20260809 06:29:02.911816 31875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:02.942041 31875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:02.946252 31875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:02.946568 31875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:02.966077 31875 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:38899
I20260809 06:29:02.966099 31965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:38899 every 8 connection(s)
I20260809 06:29:02.967496 31875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:29:02.968629 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 31875
I20260809 06:29:02.968880 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:29:02.969583 8716 external_mini_cluster.cc:1428] Setting key c606db21cc27db05267a9f22848975bc
I20260809 06:29:02.972023 31966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 06:29:02 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:29:02.985405 31966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Bootstrap starting.
I20260809 06:29:02.989401 31966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Neither blocks nor log segments found. Creating new log.
I20260809 06:29:02.990744 31966 log.cc:824] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Log is configured to *not* fsync() on all Append() calls
I20260809 06:29:02.994112 31966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: No bootstrap required, opened a new log
I20260809 06:29:02.995114 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:02.971668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37682 (local address 127.8.131.62:38899)
0809 06:29:02.972532 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:02.972547 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:02.972600 (+ 53us) server_negotiation.cc:407] Connection header received
0809 06:29:02.974866 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:02.974893 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:02.975385 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:02.975855 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:02.977203 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:02.979276 (+ 2073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:02.980826 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:02.981874 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:02.982057 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:02.986477 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:02.986488 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:02.986505 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:02.986557 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:02.990588 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:02.991361 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:02.991373 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:02.991382 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:02.991576 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:02.992024 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:02.992027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:02.992029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:02.992698 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:02.993006 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:02.993338 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 06:29:02.993773 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":384,"threads_started":1}
I20260809 06:29:03.002238 31966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:03.002564 31966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:29:03.002647 31966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 318591c3c83e4609a9c2c9ab2a600dc5, State: Initialized, Role: FOLLOWER
I20260809 06:29:03.003186 31966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:03.003427 31966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:29:03.003573 31966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:29:03.003789 31966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:29:03.006049 31966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:03.006497 31966 leader_election.cc:302] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 318591c3c83e4609a9c2c9ab2a600dc5; no voters:
I20260809 06:29:03.007077 31966 leader_election.cc:288] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:29:03.007215 31972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:29:03.008102 31972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 1 LEADER]: Becoming Leader. State: Replica: 318591c3c83e4609a9c2c9ab2a600dc5, State: Running, Role: LEADER
I20260809 06:29:03.008690 31972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:03.009366 31966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:29:03.011557 31974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 318591c3c83e4609a9c2c9ab2a600dc5. Latest consensus state: current_term: 1 leader_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } } }
I20260809 06:29:03.012100 31974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 06:29:03.012107 31973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } } }
I20260809 06:29:03.012445 31973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 06:29:03.013149 31980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:29:03.018733 31980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:29:03.027444 31980 catalog_manager.cc:1347] Generated new cluster ID: ef21f7a911a848dfb9ec29837a4bd4eb
I20260809 06:29:03.027575 31980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:29:03.048369 31980 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:29:03.049765 31980 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:29:03.060205 31980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Generated new TSK 0
I20260809 06:29:03.061095 31980 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:29:03.133104 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38899
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:29:03.471778 32000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:03.472059 32000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:03.472239 32000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:03.479380 32000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:03.479514 32000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:03.479768 32000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:29:03.489379 32000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:03.490643 32000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:03.492041 32000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:03.505277 32006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:03.505368 32009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:03.505357 32007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:03.510640 32000 server_base.cc:1034] running on GCE node
I20260809 06:29:03.511289 32000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:03.512302 32000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:03.513501 32000 hybrid_clock.cc:648] HybridClock initialized: now 1786256943513437 us; error 54 us; skew 500 ppm
I20260809 06:29:03.513806 32000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:03 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256943, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:03.517350 32000 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:29:03.519069 32000 webserver.cc:460] Webserver started at http://127.8.131.1:36491/ using document root <none> and password file <none>
I20260809 06:29:03.519963 32000 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:03.520112 32000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:03.520454 32000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:29:03.523488 32000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "e413d6c2cf534fdb916a2945e9e823c3"
format_stamp: "Formatted at 2026-08-09 06:29:03 on dist-test-slave-mjjr"
server_key: "69b99fe9f22a8469e91f107e8b34d670"
server_key_iv: "81561093bee22c02ea8f10d3d66ab398"
server_key_version: "encryptionkey@0"
I20260809 06:29:03.524340 32000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "e413d6c2cf534fdb916a2945e9e823c3"
format_stamp: "Formatted at 2026-08-09 06:29:03 on dist-test-slave-mjjr"
server_key: "69b99fe9f22a8469e91f107e8b34d670"
server_key_iv: "81561093bee22c02ea8f10d3d66ab398"
server_key_version: "encryptionkey@0"
I20260809 06:29:03.529758 32000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:29:03.533291 32016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:03.534621 32000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:29:03.534809 32000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "e413d6c2cf534fdb916a2945e9e823c3"
format_stamp: "Formatted at 2026-08-09 06:29:03 on dist-test-slave-mjjr"
server_key: "69b99fe9f22a8469e91f107e8b34d670"
server_key_iv: "81561093bee22c02ea8f10d3d66ab398"
server_key_version: "encryptionkey@0"
I20260809 06:29:03.535027 32000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:03.561455 32000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:03.565919 32000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:03.566219 32000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:03.567173 32000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:29:03.568511 32000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:29:03.568585 32000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:03.568753 32000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:29:03.568814 32000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:03.601253 32000 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:38367
I20260809 06:29:03.601302 32178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:38367 every 8 connection(s)
I20260809 06:29:03.602602 32000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:29:03.611310 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 32000
I20260809 06:29:03.611546 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:29:03.612226 8716 external_mini_cluster.cc:1428] Setting key 4393b5c3d800ae43c3353a54a11efc5a
Aug 09 06:29:03 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256943, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:29:03.623948 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:03.605200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:36977 (local address 127.8.131.62:38899)
0809 06:29:03.605679 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:03.605684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:03.606433 (+ 749us) server_negotiation.cc:407] Connection header received
0809 06:29:03.608331 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:03.608337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:03.608428 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:03.608549 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:03.610034 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:03.610761 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:03.612482 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:03.612972 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:03.613105 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:03.617028 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:03.617037 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:03.617044 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:03.617090 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:03.620667 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:03.621386 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:03.621394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:03.621401 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:03.621555 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:03.621976 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:03.621984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:03.621991 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:03.622385 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:03.622535 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:03.623184 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 06:29:03.623440 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":271,"threads_started":1}
I20260809 06:29:03.624562 32183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:03.605581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:36977)
0809 06:29:03.606264 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:03.606302 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:03.608054 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:03.608785 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:03.608793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:03.609287 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:29:03.609854 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:03.609870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:03.610916 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:03.610924 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:03.612291 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:03.612308 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:03.613225 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:03.613234 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:03.613505 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:03.614250 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:29:03.614272 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:29:03.616793 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:29:03.620815 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:03.620821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:03.620833 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:03.621250 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:03.621666 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:03.621669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:03.621671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:03.621849 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:03.622499 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:29:03.622504 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:29:03.622792 (+ 288us) client_negotiation.cc:769] Sending connection context
0809 06:29:03.623013 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 06:29:03.623380 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":266,"threads_started":1}
I20260809 06:29:03.626176 32180 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38899
I20260809 06:29:03.626516 32180 heartbeater.cc:464] Registering TS with master...
I20260809 06:29:03.627193 32180 heartbeater.cc:511] Master 127.8.131.62:38899 requested a full tablet report, sending...
I20260809 06:29:03.629266 31919 ts_manager.cc:194] Registered new tserver with Master: e413d6c2cf534fdb916a2945e9e823c3 (127.8.131.1:38367)
I20260809 06:29:03.630733 31919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:36977
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:29:03.674832 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38899
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:29:04.020421 32187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:04.020759 32187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:04.020895 32187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:04.028095 32187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:04.028203 32187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:04.028324 32187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:29:04.038002 32187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:04.039458 32187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:04.040789 32187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:04.054625 32195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:04.054625 32199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:04.054608 32196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:04.055382 32187 server_base.cc:1034] running on GCE node
I20260809 06:29:04.056062 32187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:04.057013 32187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:04.058231 32187 hybrid_clock.cc:648] HybridClock initialized: now 1786256944058154 us; error 60 us; skew 500 ppm
I20260809 06:29:04.058526 32187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:04 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:04.061757 32187 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:29:04.063129 32187 webserver.cc:460] Webserver started at http://127.8.131.2:45399/ using document root <none> and password file <none>
I20260809 06:29:04.063866 32187 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:04.063977 32187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:04.064234 32187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:29:04.067472 32187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "1517e8ba0dd6493594311e4241e59651"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "4a0c149c988f61e90aec1822d3459a83"
server_key_iv: "8f1efa949de3cf1a61195b375cbdaf48"
server_key_version: "encryptionkey@0"
I20260809 06:29:04.068259 32187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "1517e8ba0dd6493594311e4241e59651"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "4a0c149c988f61e90aec1822d3459a83"
server_key_iv: "8f1efa949de3cf1a61195b375cbdaf48"
server_key_version: "encryptionkey@0"
I20260809 06:29:04.073176 32187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:29:04.076457 32207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:04.077699 32187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 06:29:04.077829 32187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "1517e8ba0dd6493594311e4241e59651"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "4a0c149c988f61e90aec1822d3459a83"
server_key_iv: "8f1efa949de3cf1a61195b375cbdaf48"
server_key_version: "encryptionkey@0"
I20260809 06:29:04.077991 32187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:04.101183 32187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:04.105439 32187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:04.105683 32187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:04.106604 32187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:29:04.107968 32187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:29:04.108043 32187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:04.108126 32187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:29:04.108170 32187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:04.143214 32187 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:38277
I20260809 06:29:04.143292 32373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:38277 every 8 connection(s)
I20260809 06:29:04.144593 32187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:29:04.153369 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 32187
I20260809 06:29:04.153573 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:29:04.154039 8716 external_mini_cluster.cc:1428] Setting key 60263eb6b2a54bc320c63208f96fb0a9
Aug 09 06:29:04 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:29:04.167019 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.147146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:38973 (local address 127.8.131.62:38899)
0809 06:29:04.147579 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:04.147584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:04.148547 (+ 963us) server_negotiation.cc:407] Connection header received
0809 06:29:04.150377 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:04.150392 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:04.150482 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:04.150605 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:04.152134 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.152884 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.154568 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.155167 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.155307 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.159512 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:04.159520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:04.159527 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:04.159570 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:04.163045 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:04.164053 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:04.164060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:04.164066 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:04.164210 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:04.164718 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:04.164725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:04.164730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:04.165097 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:04.165232 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:04.166126 (+ 894us) server_negotiation.cc:299] Negotiation successful
0809 06:29:04.166537 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":328,"thread_start_us":222,"threads_started":1}
I20260809 06:29:04.168053 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.147695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:38973)
0809 06:29:04.148383 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:04.148429 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:04.150115 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:04.150832 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:04.150841 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:04.151334 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:29:04.151966 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.151984 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.153069 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.153073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.154422 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.154434 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.155405 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.155413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.155760 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.156482 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:29:04.156504 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:29:04.159241 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:29:04.163203 (+ 3962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:04.163215 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:04.163236 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:04.163927 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:04.164324 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:04.164331 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:04.164337 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:04.164600 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:04.165234 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:29:04.165245 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:29:04.165715 (+ 470us) client_negotiation.cc:769] Sending connection context
0809 06:29:04.166020 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 06:29:04.166623 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":261,"threads_started":1}
I20260809 06:29:04.169564 32374 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38899
I20260809 06:29:04.169883 32374 heartbeater.cc:464] Registering TS with master...
I20260809 06:29:04.170579 32374 heartbeater.cc:511] Master 127.8.131.62:38899 requested a full tablet report, sending...
I20260809 06:29:04.172026 31919 ts_manager.cc:194] Registered new tserver with Master: 1517e8ba0dd6493594311e4241e59651 (127.8.131.2:38277)
I20260809 06:29:04.172839 31919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:38973
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:29:04.215212 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38899
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:29:04.548003 32382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:04.548324 32382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:04.548509 32382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:04.555672 32382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:04.555830 32382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:04.556030 32382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:29:04.565652 32382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:04.566972 32382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:04.568337 32382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:04.581552 32391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:04.581632 32392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:04.581596 32399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:04.582609 32382 server_base.cc:1034] running on GCE node
I20260809 06:29:04.583240 32382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:04.584193 32382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:04.585430 32382 hybrid_clock.cc:648] HybridClock initialized: now 1786256944585390 us; error 46 us; skew 500 ppm
I20260809 06:29:04.585737 32382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:04 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:04.588804 32382 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:29:04.590195 32382 webserver.cc:460] Webserver started at http://127.8.131.3:46273/ using document root <none> and password file <none>
I20260809 06:29:04.590859 32382 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:04.591017 32382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:04.591339 32382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:29:04.594264 32382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "322b47beca114cc59490353149f235c6"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "023331d978aad80e389988df55d7cea3"
server_key_iv: "3b566a00368b1ba16ebbc8c4c02ea718"
server_key_version: "encryptionkey@0"
I20260809 06:29:04.595074 32382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "322b47beca114cc59490353149f235c6"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "023331d978aad80e389988df55d7cea3"
server_key_iv: "3b566a00368b1ba16ebbc8c4c02ea718"
server_key_version: "encryptionkey@0"
I20260809 06:29:04.600274 32382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 06:29:04.603832 32410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:04.605237 32382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:29:04.605413 32382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "322b47beca114cc59490353149f235c6"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "023331d978aad80e389988df55d7cea3"
server_key_iv: "3b566a00368b1ba16ebbc8c4c02ea718"
server_key_version: "encryptionkey@0"
I20260809 06:29:04.605612 32382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:04.624034 32382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:04.627935 32382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:04.628230 32382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:04.629163 32382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:29:04.630491 32382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:29:04.630596 32382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:04.630746 32382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:29:04.630820 32382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:04.633494 32180 heartbeater.cc:503] Master 127.8.131.62:38899 was elected leader, sending a full tablet report...
I20260809 06:29:04.664796 32382 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:46851
I20260809 06:29:04.664834 32583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:46851 every 8 connection(s)
I20260809 06:29:04.666208 32382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:29:04.670374 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 32382
I20260809 06:29:04.670578 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:29:04.671020 8716 external_mini_cluster.cc:1428] Setting key 28191bf35280f22412b3a2f57ffde489
Aug 09 06:29:04 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:29:04.686048 32589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.668765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:41117 (local address 127.8.131.62:38899)
0809 06:29:04.669276 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:04.669282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:04.670107 (+ 825us) server_negotiation.cc:407] Connection header received
0809 06:29:04.672101 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:04.672106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:04.672192 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:04.672319 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:04.673767 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.674551 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.676145 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.676546 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.676681 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.680096 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:04.680101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:04.680104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:04.680136 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:04.682831 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:04.683483 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:04.683486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:04.683488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:04.683600 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:04.684037 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:04.684040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:04.684042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:04.684411 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:04.684558 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:04.685435 (+ 877us) server_negotiation.cc:299] Negotiation successful
0809 06:29:04.685658 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":226,"threads_started":1}
I20260809 06:29:04.686919 32590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.669282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:41117)
0809 06:29:04.669955 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:04.669992 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:04.671833 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:04.672555 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:04.672563 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:04.673064 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:29:04.673597 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.673613 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.674713 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.674723 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.675999 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.676011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.676780 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.676787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.677022 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.677702 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:29:04.677729 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:29:04.679888 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:29:04.682968 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:04.682975 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:04.682985 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:04.683373 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:04.683745 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:04.683748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:04.683750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:04.683920 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:04.684568 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:29:04.684578 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:29:04.685002 (+ 424us) client_negotiation.cc:769] Sending connection context
0809 06:29:04.685303 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 06:29:04.685808 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":259,"threads_started":1}
I20260809 06:29:04.688198 32585 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38899
I20260809 06:29:04.688509 32585 heartbeater.cc:464] Registering TS with master...
I20260809 06:29:04.689213 32585 heartbeater.cc:511] Master 127.8.131.62:38899 requested a full tablet report, sending...
I20260809 06:29:04.690580 31919 ts_manager.cc:194] Registered new tserver with Master: 322b47beca114cc59490353149f235c6 (127.8.131.3:46851)
I20260809 06:29:04.691319 31919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:41117
I20260809 06:29:04.698239 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:29:04.715719 32589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.702983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.8.131.62:38899)
0809 06:29:04.703205 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:04.703214 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:04.703530 (+ 316us) server_negotiation.cc:407] Connection header received
0809 06:29:04.703715 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:04.703718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:04.703778 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:04.703859 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:04.705253 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.705953 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.707258 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.707734 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.707825 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.710476 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:04.710480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:04.710483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:04.710512 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:04.712999 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:04.713618 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:04.713621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:04.713623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:04.713734 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:04.714162 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:04.714168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:04.714171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:04.714511 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:04.714768 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:04.715097 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 06:29:04.715343 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 06:29:04.724244 31919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:29:04.731103 31919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:29:04.765029 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.743552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46734 (local address 127.8.131.1:38367)
0809 06:29:04.757216 (+ 13664us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:04.757221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:04.757243 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:29:04.757317 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:04.757322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:04.757492 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:04.757690 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:04.758423 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.760066 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.762365 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.763807 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.763968 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.764216 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:04.764495 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 06:29:04.764662 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13494,"thread_start_us":13336,"threads_started":1}
I20260809 06:29:04.765166 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.743290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38367 (local address 127.0.0.1:46734)
0809 06:29:04.744103 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:04.744152 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:04.744349 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:04.757680 (+ 13331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:04.757690 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:04.757714 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:04.758231 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.758247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.760212 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.760217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.762115 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.762127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.764000 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.764006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.764214 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.764276 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 06:29:04.764460 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 06:29:04.764662 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":149,"thread_start_us":12085,"threads_started":1}
I20260809 06:29:04.765702 32609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.744580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:38277 (local address 127.0.0.1:56850)
0809 06:29:04.755993 (+ 11413us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:04.756016 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:04.756153 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:04.757108 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:04.757119 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:04.757153 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:04.757632 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.757647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.759980 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.759986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.762845 (+ 2859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.762861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.764783 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.764789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.764991 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.765046 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 06:29:04.765121 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 06:29:04.765251 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11260,"mutex_wait_us":247,"thread_start_us":11124,"threads_started":1}
I20260809 06:29:04.765904 32606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.743320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:46851 (local address 127.0.0.1:39838)
0809 06:29:04.755696 (+ 12376us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:04.755717 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:04.755880 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:04.757369 (+ 1489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:04.757373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:04.757394 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:04.757858 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.757870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.759884 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.759890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.762845 (+ 2955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.762865 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.765076 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.765084 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.765247 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.765274 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 06:29:04.765353 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 06:29:04.765482 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12205,"mutex_wait_us":166,"thread_start_us":418,"threads_started":1}
I20260809 06:29:04.766566 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.744897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.8.131.2:38277)
0809 06:29:04.755570 (+ 10673us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:04.755583 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:04.756301 (+ 718us) server_negotiation.cc:407] Connection header received
0809 06:29:04.756651 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:04.756658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:04.756846 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:04.757861 (+ 1015us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:04.758142 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.759843 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.763030 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.764552 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.765776 (+ 1224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.765951 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:04.766016 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 06:29:04.766193 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10451,"thread_start_us":10327,"threads_started":1}
I20260809 06:29:04.766866 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.744073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.8.131.3:46851)
0809 06:29:04.756360 (+ 12287us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:04.756368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:04.756398 (+ 30us) server_negotiation.cc:407] Connection header received
0809 06:29:04.756525 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:04.756533 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:04.756849 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:04.757065 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:04.758114 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.759732 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.763030 (+ 3298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.764544 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.764682 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.764929 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:04.766287 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0809 06:29:04.766516 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12097,"thread_start_us":12333,"threads_started":1}
I20260809 06:29:04.770009 32086 tablet_service.cc:1467] Processing CreateTablet for tablet 42cd2c38ec484ddca35237d27053cdf3 (DEFAULT_TABLE table=test-table [id=60dd091d69cc4a779351a6834e1ca636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:29:04.770228 32471 tablet_service.cc:1467] Processing CreateTablet for tablet 42cd2c38ec484ddca35237d27053cdf3 (DEFAULT_TABLE table=test-table [id=60dd091d69cc4a779351a6834e1ca636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:29:04.770221 32276 tablet_service.cc:1467] Processing CreateTablet for tablet 42cd2c38ec484ddca35237d27053cdf3 (DEFAULT_TABLE table=test-table [id=60dd091d69cc4a779351a6834e1ca636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:29:04.771440 32086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42cd2c38ec484ddca35237d27053cdf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:29:04.771548 32276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42cd2c38ec484ddca35237d27053cdf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:29:04.771545 32471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42cd2c38ec484ddca35237d27053cdf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:29:04.783279 32611 tablet_bootstrap.cc:492] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: Bootstrap starting.
I20260809 06:29:04.785913 32612 tablet_bootstrap.cc:492] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: Bootstrap starting.
I20260809 06:29:04.786966 32611 tablet_bootstrap.cc:654] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: Neither blocks nor log segments found. Creating new log.
I20260809 06:29:04.788381 32611 log.cc:824] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:29:04.789523 32612 tablet_bootstrap.cc:654] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: Neither blocks nor log segments found. Creating new log.
I20260809 06:29:04.791172 32612 log.cc:824] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: Log is configured to *not* fsync() on all Append() calls
I20260809 06:29:04.792615 32613 tablet_bootstrap.cc:492] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: Bootstrap starting.
I20260809 06:29:04.794500 32611 tablet_bootstrap.cc:492] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: No bootstrap required, opened a new log
I20260809 06:29:04.794591 32612 tablet_bootstrap.cc:492] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: No bootstrap required, opened a new log
I20260809 06:29:04.794790 32611 ts_tablet_manager.cc:1397] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260809 06:29:04.794881 32612 ts_tablet_manager.cc:1397] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 06:29:04.796257 32613 tablet_bootstrap.cc:654] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: Neither blocks nor log segments found. Creating new log.
I20260809 06:29:04.797605 32613 log.cc:824] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: Log is configured to *not* fsync() on all Append() calls
I20260809 06:29:04.801402 32613 tablet_bootstrap.cc:492] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: No bootstrap required, opened a new log
I20260809 06:29:04.801720 32613 ts_tablet_manager.cc:1397] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 06:29:04.803148 32611 raft_consensus.cc:348] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } }
I20260809 06:29:04.803532 32611 raft_consensus.cc:374] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:29:04.803699 32611 raft_consensus.cc:729] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e413d6c2cf534fdb916a2945e9e823c3, State: Initialized, Role: FOLLOWER
I20260809 06:29:04.804351 32611 consensus_queue.cc:260] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } }
I20260809 06:29:04.805580 32612 raft_consensus.cc:348] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } }
I20260809 06:29:04.806023 32611 ts_tablet_manager.cc:1428] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: Time spent starting tablet: real 0.011s user 0.008s sys 0.005s
I20260809 06:29:04.806048 32612 raft_consensus.cc:374] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:29:04.806146 32612 raft_consensus.cc:729] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 322b47beca114cc59490353149f235c6, State: Initialized, Role: FOLLOWER
I20260809 06:29:04.806926 32612 consensus_queue.cc:260] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } }
I20260809 06:29:04.808115 32585 heartbeater.cc:503] Master 127.8.131.62:38899 was elected leader, sending a full tablet report...
I20260809 06:29:04.808858 32612 ts_tablet_manager.cc:1428] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 06:29:04.813010 32613 raft_consensus.cc:348] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } }
I20260809 06:29:04.813393 32613 raft_consensus.cc:374] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:29:04.813524 32613 raft_consensus.cc:729] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1517e8ba0dd6493594311e4241e59651, State: Initialized, Role: FOLLOWER
I20260809 06:29:04.814051 32613 consensus_queue.cc:260] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } }
I20260809 06:29:04.815069 32374 heartbeater.cc:503] Master 127.8.131.62:38899 was elected leader, sending a full tablet report...
I20260809 06:29:04.815802 32613 ts_tablet_manager.cc:1428] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260809 06:29:04.855259 32181 tablet.cc:2369] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:29:04.896600 32377 tablet.cc:2369] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:29:04.912081 32619 raft_consensus.cc:482] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:29:04.912442 32619 raft_consensus.cc:504] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } }
I20260809 06:29:04.914978 32619 leader_election.cc:288] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 322b47beca114cc59490353149f235c6 (127.8.131.3:46851), 1517e8ba0dd6493594311e4241e59651 (127.8.131.2:38277)
W20260809 06:29:04.918330 32588 tablet.cc:2369] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:29:04.924966 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.915420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:47269 (local address 127.8.131.3:46851)
0809 06:29:04.915603 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:04.915609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:04.916020 (+ 411us) server_negotiation.cc:407] Connection header received
0809 06:29:04.916396 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:04.916405 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:04.916507 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:04.916585 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:04.918552 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.919839 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.922205 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.923538 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.923675 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.923799 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:04.924463 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 06:29:04.924623 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 06:29:04.925436 32625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.916070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:38277 (local address 127.8.131.1:38517)
0809 06:29:04.919055 (+ 2985us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:04.919074 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:04.919198 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:04.919535 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:04.919540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:04.919568 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:04.919941 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.919955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.921089 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.921094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.923265 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.923282 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.924531 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.924537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.924722 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.924753 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 06:29:04.924868 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 06:29:04.924991 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2864,"mutex_wait_us":57,"thread_start_us":2651,"threads_started":1}
I20260809 06:29:04.925593 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.916248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:38517 (local address 127.8.131.2:38277)
0809 06:29:04.916431 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:04.916438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:04.919183 (+ 2745us) server_negotiation.cc:407] Connection header received
0809 06:29:04.919339 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:04.919344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:04.919423 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:04.919525 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:04.920146 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.920945 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.923443 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.924294 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.924453 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.924555 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:04.924914 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 06:29:04.925326 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:29:04.925618 32622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:04.915180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:46851 (local address 127.8.131.1:47269)
0809 06:29:04.915897 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:04.915921 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:04.916157 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:04.916888 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:04.916895 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:04.916932 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:04.918353 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.918369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.920010 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.920017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.922013 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:04.922028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:04.924005 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:04.924013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:04.924239 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:04.924281 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 06:29:04.925161 (+ 880us) client_negotiation.cc:240] Negotiation successful
0809 06:29:04.925277 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":3,"thread_start_us":452,"threads_started":1}
I20260809 06:29:04.926580 32304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cd2c38ec484ddca35237d27053cdf3" candidate_uuid: "e413d6c2cf534fdb916a2945e9e823c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1517e8ba0dd6493594311e4241e59651" is_pre_election: true
I20260809 06:29:04.926606 32509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cd2c38ec484ddca35237d27053cdf3" candidate_uuid: "e413d6c2cf534fdb916a2945e9e823c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "322b47beca114cc59490353149f235c6" is_pre_election: true
I20260809 06:29:04.927172 32304 raft_consensus.cc:2454] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e413d6c2cf534fdb916a2945e9e823c3 in term 0.
I20260809 06:29:04.927176 32509 raft_consensus.cc:2454] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e413d6c2cf534fdb916a2945e9e823c3 in term 0.
I20260809 06:29:04.928007 32017 leader_election.cc:302] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1517e8ba0dd6493594311e4241e59651, e413d6c2cf534fdb916a2945e9e823c3; no voters:
I20260809 06:29:04.928503 32619 raft_consensus.cc:2781] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:29:04.928674 32619 raft_consensus.cc:482] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:29:04.928778 32619 raft_consensus.cc:3037] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:29:04.932364 32619 raft_consensus.cc:504] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } }
I20260809 06:29:04.933164 32619 leader_election.cc:288] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [CANDIDATE]: Term 1 election: Requested vote from peers 322b47beca114cc59490353149f235c6 (127.8.131.3:46851), 1517e8ba0dd6493594311e4241e59651 (127.8.131.2:38277)
I20260809 06:29:04.933604 32509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cd2c38ec484ddca35237d27053cdf3" candidate_uuid: "e413d6c2cf534fdb916a2945e9e823c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "322b47beca114cc59490353149f235c6"
I20260809 06:29:04.933718 32304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42cd2c38ec484ddca35237d27053cdf3" candidate_uuid: "e413d6c2cf534fdb916a2945e9e823c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1517e8ba0dd6493594311e4241e59651"
I20260809 06:29:04.933815 32509 raft_consensus.cc:3037] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:29:04.933966 32304 raft_consensus.cc:3037] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:29:04.937649 32509 raft_consensus.cc:2454] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e413d6c2cf534fdb916a2945e9e823c3 in term 1.
I20260809 06:29:04.938081 32304 raft_consensus.cc:2454] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e413d6c2cf534fdb916a2945e9e823c3 in term 1.
I20260809 06:29:04.938181 32017 leader_election.cc:302] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 322b47beca114cc59490353149f235c6, e413d6c2cf534fdb916a2945e9e823c3; no voters:
I20260809 06:29:04.938872 32619 raft_consensus.cc:2781] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:29:04.939461 32619 raft_consensus.cc:686] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 LEADER]: Becoming Leader. State: Replica: e413d6c2cf534fdb916a2945e9e823c3, State: Running, Role: LEADER
I20260809 06:29:04.940136 32619 consensus_queue.cc:237] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } }
I20260809 06:29:04.945958 31918 catalog_manager.cc:5495] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e413d6c2cf534fdb916a2945e9e823c3 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "e413d6c2cf534fdb916a2945e9e823c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } health_report { overall_health: UNKNOWN } } }
I20260809 06:29:05.010234 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:05.004623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46748 (local address 127.8.131.1:38367)
0809 06:29:05.004856 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:05.004862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:05.004884 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:29:05.005009 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:05.005012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:05.005103 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:05.005268 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:29:05.005662 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:05.006385 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:05.007906 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:05.008317 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:05.008453 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:05.009051 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:29:05.009605 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:29:05.009821 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:05.009871 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 06:29:05.009962 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 06:29:05.018664 32304 raft_consensus.cc:1261] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [term 1 FOLLOWER]: Refusing update from remote peer e413d6c2cf534fdb916a2945e9e823c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:29:05.018735 32509 raft_consensus.cc:1261] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [term 1 FOLLOWER]: Refusing update from remote peer e413d6c2cf534fdb916a2945e9e823c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:29:05.019656 32626 consensus_queue.cc:1035] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:29:05.020047 32619 consensus_queue.cc:1035] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:29:05.031713 32630 mvcc.cc:205] Tried to move back new op lower bound from 7316508446786961408 to 7316508446478757888. Current Snapshot: MvccSnapshot[applied={T|T < 7316508446786961408}]
I20260809 06:29:05.046288 31919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:29:05.046590 31919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:29:05.050351 31919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Sending DeleteTablet for 3 replicas of tablet 42cd2c38ec484ddca35237d27053cdf3
I20260809 06:29:05.051631 32086 tablet_service.cc:1514] Processing DeleteTablet for tablet 42cd2c38ec484ddca35237d27053cdf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:29:05 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:46734
I20260809 06:29:05.051908 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 32000
I20260809 06:29:05.051786 32471 tablet_service.cc:1514] Processing DeleteTablet for tablet 42cd2c38ec484ddca35237d27053cdf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:29:05 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:39838
I20260809 06:29:05.052703 32276 tablet_service.cc:1514] Processing DeleteTablet for tablet 42cd2c38ec484ddca35237d27053cdf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:29:05 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:56850
I20260809 06:29:05.064591 32649 tablet_replica.cc:330] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: stopping tablet replica
I20260809 06:29:05.065127 32653 tablet_replica.cc:330] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: stopping tablet replica
I20260809 06:29:05.065217 32649 raft_consensus.cc:2229] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:29:05.065757 32653 raft_consensus.cc:2229] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:29:05.066215 32649 raft_consensus.cc:2258] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:29:05.066335 32653 raft_consensus.cc:2258] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:29:05.067291 32650 tablet_replica.cc:330] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: stopping tablet replica
I20260809 06:29:05.067780 32650 raft_consensus.cc:2229] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:29:05.068413 32650 raft_consensus.cc:2258] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:29:05.069859 32653 ts_tablet_manager.cc:1905] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:29:05.070628 32650 ts_tablet_manager.cc:1905] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:29:05.071066 32649 ts_tablet_manager.cc:1905] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:29:05.076977 32653 ts_tablet_manager.cc:1918] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:29:05.077145 32653 log.cc:1196] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/42cd2c38ec484ddca35237d27053cdf3
I20260809 06:29:05.077683 32653 ts_tablet_manager.cc:1939] T 42cd2c38ec484ddca35237d27053cdf3 P 1517e8ba0dd6493594311e4241e59651: Deleting consensus metadata
I20260809 06:29:05.077955 32650 ts_tablet_manager.cc:1918] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:29:05.078184 32650 log.cc:1196] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/42cd2c38ec484ddca35237d27053cdf3
I20260809 06:29:05.078806 32650 ts_tablet_manager.cc:1939] T 42cd2c38ec484ddca35237d27053cdf3 P 322b47beca114cc59490353149f235c6: Deleting consensus metadata
I20260809 06:29:05.079726 31896 catalog_manager.cc:4841] TS 1517e8ba0dd6493594311e4241e59651 (127.8.131.2:38277): tablet 42cd2c38ec484ddca35237d27053cdf3 (table test-table [id=60dd091d69cc4a779351a6834e1ca636]) successfully deleted
I20260809 06:29:05.080721 31896 catalog_manager.cc:4841] TS 322b47beca114cc59490353149f235c6 (127.8.131.3:46851): tablet 42cd2c38ec484ddca35237d27053cdf3 (table test-table [id=60dd091d69cc4a779351a6834e1ca636]) successfully deleted
I20260809 06:29:05.082828 32649 ts_tablet_manager.cc:1918] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:29:05.083034 32649 log.cc:1196] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/42cd2c38ec484ddca35237d27053cdf3
I20260809 06:29:05.083515 32649 ts_tablet_manager.cc:1939] T 42cd2c38ec484ddca35237d27053cdf3 P e413d6c2cf534fdb916a2945e9e823c3: Deleting consensus metadata
I20260809 06:29:05.085144 31899 catalog_manager.cc:4841] TS e413d6c2cf534fdb916a2945e9e823c3 (127.8.131.1:38367): tablet 42cd2c38ec484ddca35237d27053cdf3 (table test-table [id=60dd091d69cc4a779351a6834e1ca636]) successfully deleted
Aug 09 06:29:05 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:29:05.092692 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:05.053053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.8.131.1:38367)
0809 06:29:05.053257 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:05.053266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:05.063999 (+ 10733us) server_negotiation.cc:407] Connection header received
0809 06:29:05.067022 (+ 3023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:05.067027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:05.067122 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:05.067207 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:05.071778 (+ 4571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:05.072988 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:05.079767 (+ 6779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:05.080526 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:05.080651 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:05.086498 (+ 5847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:05.086510 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:05.086524 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:05.086566 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:05.089662 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:05.090422 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:05.090430 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:05.090432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:05.090590 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:05.091024 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:05.091028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:05.091030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:05.091390 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:05.091748 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:05.092088 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 06:29:05.092347 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:29:05.093408 32170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:05.195434 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 32000
I20260809 06:29:05.222002 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 32187
Aug 09 06:29:05 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:29:05.237236 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:05.222873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56858 (local address 127.8.131.2:38277)
0809 06:29:05.223053 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:05.223058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:05.223075 (+ 17us) server_negotiation.cc:407] Connection header received
0809 06:29:05.223130 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:05.223133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:05.223192 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:05.223285 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:05.224643 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:05.225406 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:05.226723 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:05.227098 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:05.227189 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:05.231573 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:05.231586 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:05.231600 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:05.231634 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:05.234313 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:05.235073 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:05.235080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:05.235082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:05.235205 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:05.235584 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:05.235590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:05.235595 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:05.236033 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:05.236342 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:05.236651 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 06:29:05.236869 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:29:05.237946 32367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:05.327893 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 32187
I20260809 06:29:05.349606 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 32382
Aug 09 06:29:05 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:29:05.364312 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:05.350460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.8.131.3:46851)
0809 06:29:05.350650 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:05.350654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:05.350673 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:29:05.350822 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:05.350826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:05.350890 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:05.350980 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:05.352392 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:05.353097 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:05.354389 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:05.354762 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:05.354851 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:05.359198 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:05.359212 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:05.359227 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:05.359269 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:05.361879 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:05.362527 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:05.362534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:05.362536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:05.362665 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:05.362979 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:05.362982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:05.362984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:05.363309 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:05.363536 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:05.363808 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 06:29:05.363957 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:29:05.364979 32575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:05.455539 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 32382
I20260809 06:29:05.478329 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 31875
I20260809 06:29:05.479523 31960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:05.591711 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 31875
I20260809 06:29:25.619722 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38899
--webserver_interface=127.8.131.62
--webserver_port=34311
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:38899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:29:25.972254 32661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:25.972543 32661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:25.972707 32661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:25.980031 32661 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 06:29:25.980134 32661 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 06:29:25.980232 32661 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 06:29:25.980329 32661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:29:25.980425 32661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:25.980552 32661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:25.980644 32661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:29:25.980729 32661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:29:25.990926 32661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:38899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=34311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:25.992233 32661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:25.993585 32661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:26.019515 32677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:26.019585 32671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:26.019546 32669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:26.019901 32661 server_base.cc:1034] running on GCE node
I20260809 06:29:26.021204 32661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:26.022760 32661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:26.024025 32661 hybrid_clock.cc:648] HybridClock initialized: now 1786256966023947 us; error 48 us; skew 500 ppm
I20260809 06:29:26.024425 32661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:26 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256966, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:26.028116 32661 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:29:26.029775 32661 webserver.cc:460] Webserver started at http://127.8.131.62:34311/ using document root <none> and password file <none>
I20260809 06:29:26.030555 32661 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:26.030706 32661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:26.036710 32661 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 06:29:26.039631 32686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:26.041031 32661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:29:26.041216 32661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "318591c3c83e4609a9c2c9ab2a600dc5"
format_stamp: "Formatted at 2026-08-09 06:29:02 on dist-test-slave-mjjr"
server_key: "ec2cf10be60df12f0c50b508aea35f96"
server_key_iv: "49d97546cdb01ffb800d13fbf5b903df"
server_key_version: "encryptionkey@0"
I20260809 06:29:26.042279 32661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:26.064369 32661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:26.068589 32661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:26.068902 32661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:26.088387 32661 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:38899
I20260809 06:29:26.088436 306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:38899 every 8 connection(s)
I20260809 06:29:26.089915 32661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:29:26.090729 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 32661
I20260809 06:29:26.091931 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:38367
--local_ip_for_outbound_sockets=127.8.131.1
--tserver_master_addrs=127.8.131.62:38899
--webserver_port=36491
--webserver_interface=127.8.131.1
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:29:26.097339 307 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:29:26.100968 307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Bootstrap starting.
I20260809 06:29:26.129505 307 log.cc:824] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Log is configured to *not* fsync() on all Append() calls
I20260809 06:29:26.145048 307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:29:26.145820 307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Bootstrap complete.
I20260809 06:29:26.160007 307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:26.160933 307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 318591c3c83e4609a9c2c9ab2a600dc5, State: Initialized, Role: FOLLOWER
I20260809 06:29:26.161617 307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:26.161880 307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:29:26.162024 307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:29:26.162202 307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 06:29:26.166090 307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:26.166654 307 leader_election.cc:302] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 318591c3c83e4609a9c2c9ab2a600dc5; no voters:
I20260809 06:29:26.167384 307 leader_election.cc:288] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 06:29:26.167500 313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 06:29:26.168776 313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 2 LEADER]: Becoming Leader. State: Replica: 318591c3c83e4609a9c2c9ab2a600dc5, State: Running, Role: LEADER
I20260809 06:29:26.169497 313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:26.169740 307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:29:26.172197 318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 318591c3c83e4609a9c2c9ab2a600dc5. Latest consensus state: current_term: 2 leader_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } } }
I20260809 06:29:26.172590 318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 06:29:26.173153 317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } } }
I20260809 06:29:26.173482 317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 06:29:26.178756 322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:29:26.188848 322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:29:26.190925 322 catalog_manager.cc:1259] Loaded cluster ID: ef21f7a911a848dfb9ec29837a4bd4eb
I20260809 06:29:26.191128 322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:29:26.196419 322 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:29:26.198508 322 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Loaded TSK: 0
I20260809 06:29:26.207294 337 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 06:29:26.227607 322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Generated new TSK 1
I20260809 06:29:26.227998 322 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 06:29:26.485879 309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:26.486225 309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:26.486409 309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:26.493996 309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:26.494166 309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:26.494347 309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:29:26.504566 309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:38367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=36491
--tserver_master_addrs=127.8.131.62:38899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:26.506006 309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:26.507373 309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:26.520553 346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:26.521057 345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:26.521703 350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:26.522881 309 server_base.cc:1034] running on GCE node
I20260809 06:29:26.523609 309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:26.524814 309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:26.526075 309 hybrid_clock.cc:648] HybridClock initialized: now 1786256966525985 us; error 94 us; skew 500 ppm
I20260809 06:29:26.526458 309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:26 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256966, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:26.530586 309 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:29:26.532289 309 webserver.cc:460] Webserver started at http://127.8.131.1:36491/ using document root <none> and password file <none>
I20260809 06:29:26.533092 309 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:26.533238 309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:26.540277 309 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:29:26.543534 363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:26.545120 309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:29:26.545285 309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "e413d6c2cf534fdb916a2945e9e823c3"
format_stamp: "Formatted at 2026-08-09 06:29:03 on dist-test-slave-mjjr"
server_key: "69b99fe9f22a8469e91f107e8b34d670"
server_key_iv: "81561093bee22c02ea8f10d3d66ab398"
server_key_version: "encryptionkey@0"
I20260809 06:29:26.546680 309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:26.568094 309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:26.573451 309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:26.573733 309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:26.574864 309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:29:26.576642 309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:29:26.576764 309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:26.576891 309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:29:26.576969 309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:26.611840 309 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:38367
I20260809 06:29:26.611965 531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:38367 every 8 connection(s)
I20260809 06:29:26.613333 309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:29:26.617626 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 309
I20260809 06:29:26.618289 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:38277
--local_ip_for_outbound_sockets=127.8.131.2
--tserver_master_addrs=127.8.131.62:38899
--webserver_port=45399
--webserver_interface=127.8.131.2
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:29:26 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256966, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:29:26.637941 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:26.616295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:41737 (local address 127.8.131.62:38899)
0809 06:29:26.616916 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:26.616927 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:26.617210 (+ 283us) server_negotiation.cc:407] Connection header received
0809 06:29:26.619365 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:26.619388 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:26.619856 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:26.620277 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:26.621724 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:26.623767 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:26.625308 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:26.626438 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:26.626641 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:26.630298 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:26.630306 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:26.630319 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:26.630381 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:26.633394 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:26.634379 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:26.634385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:26.634389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:26.634541 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:26.635071 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:26.635074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:26.635076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:26.635520 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:26.635718 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:26.636641 (+ 923us) server_negotiation.cc:299] Negotiation successful
0809 06:29:26.636943 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":295,"threads_started":1}
I20260809 06:29:26.638404 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:26.616233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:41737)
0809 06:29:26.617026 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:26.617070 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:26.618758 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:26.620360 (+ 1602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:26.620367 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:26.620858 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:29:26.621367 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:26.621395 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:26.623932 (+ 2537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:26.623936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:26.625133 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:26.625145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:26.626679 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:26.626684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:26.626859 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:26.627541 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:29:26.627570 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:29:26.630000 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:29:26.633544 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:26.633555 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:26.633573 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:26.634244 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:26.634663 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:26.634670 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:26.634677 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:26.634946 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:26.635716 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:29:26.635726 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:29:26.636146 (+ 420us) client_negotiation.cc:769] Sending connection context
0809 06:29:26.636455 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 06:29:26.637050 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":342,"threads_started":1}
I20260809 06:29:26.641402 532 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38899
I20260809 06:29:26.641707 532 heartbeater.cc:464] Registering TS with master...
I20260809 06:29:26.642441 532 heartbeater.cc:511] Master 127.8.131.62:38899 requested a full tablet report, sending...
I20260809 06:29:26.645129 32712 ts_manager.cc:194] Registered new tserver with Master: e413d6c2cf534fdb916a2945e9e823c3 (127.8.131.1:38367)
I20260809 06:29:26.647526 32712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:41737
W20260809 06:29:26.960381 536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:26.960646 536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:26.960739 536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:26.967828 536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:26.967941 536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:26.968053 536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:29:26.977792 536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:38277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=45399
--tserver_master_addrs=127.8.131.62:38899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:26.979085 536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:26.980468 536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:26.993573 544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:26.993531 543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:26.993480 547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:26.994418 536 server_base.cc:1034] running on GCE node
I20260809 06:29:26.995002 536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:26.995997 536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:26.997233 536 hybrid_clock.cc:648] HybridClock initialized: now 1786256966997186 us; error 58 us; skew 500 ppm
I20260809 06:29:26.997537 536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:26 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256966, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:27.000666 536 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:29:27.002163 536 webserver.cc:460] Webserver started at http://127.8.131.2:45399/ using document root <none> and password file <none>
I20260809 06:29:27.002876 536 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:27.003049 536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:27.008234 536 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:29:27.010798 560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:27.012183 536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:29:27.012372 536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "1517e8ba0dd6493594311e4241e59651"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "4a0c149c988f61e90aec1822d3459a83"
server_key_iv: "8f1efa949de3cf1a61195b375cbdaf48"
server_key_version: "encryptionkey@0"
I20260809 06:29:27.013321 536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:27.029412 536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:27.033301 536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:27.033584 536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:27.034543 536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:29:27.035995 536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:29:27.036114 536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:27.036245 536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:29:27.036345 536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:27.070245 536 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:38277
I20260809 06:29:27.070343 731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:38277 every 8 connection(s)
I20260809 06:29:27.071834 536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:29:27.081131 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 536
I20260809 06:29:27.081801 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:46851
--local_ip_for_outbound_sockets=127.8.131.3
--tserver_master_addrs=127.8.131.62:38899
--webserver_port=46273
--webserver_interface=127.8.131.3
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 06:29:27 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256966, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:29:27.092962 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.074349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:34927 (local address 127.8.131.62:38899)
0809 06:29:27.074515 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.074519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.075551 (+ 1032us) server_negotiation.cc:407] Connection header received
0809 06:29:27.077457 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.077463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.077557 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.077674 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:27.079202 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.080060 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.081680 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.082176 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.082311 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.086102 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:27.086107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:27.086110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:27.086143 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:27.089270 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.090241 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.090246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.090250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.090396 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.090928 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.090932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.090936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.091343 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:27.091500 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.092382 (+ 882us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.092623 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 06:29:27.093945 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.074705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:34927)
0809 06:29:27.075392 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.075434 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:27.077174 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:27.077935 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:27.077943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:27.078438 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:29:27.078999 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.079015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.080226 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.080230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.081525 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.081536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.082408 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.082417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.082649 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.083369 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:29:27.083391 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:29:27.085875 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:29:27.089438 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:27.089450 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:27.089467 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:27.090108 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:27.090520 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:27.090528 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:27.090534 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:27.090801 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:27.091486 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:29:27.091496 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:29:27.091954 (+ 458us) client_negotiation.cc:769] Sending connection context
0809 06:29:27.092254 (+ 300us) client_negotiation.cc:240] Negotiation successful
0809 06:29:27.092816 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":276,"threads_started":1}
I20260809 06:29:27.095603 732 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38899
I20260809 06:29:27.095937 732 heartbeater.cc:464] Registering TS with master...
I20260809 06:29:27.096758 732 heartbeater.cc:511] Master 127.8.131.62:38899 requested a full tablet report, sending...
I20260809 06:29:27.098542 32712 ts_manager.cc:194] Registered new tserver with Master: 1517e8ba0dd6493594311e4241e59651 (127.8.131.2:38277)
I20260809 06:29:27.099896 32712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:34927
W20260809 06:29:27.422415 735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:27.422659 735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:27.422746 735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:27.429965 735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:27.430060 735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:27.430164 735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:29:27.439409 735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:46851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=46273
--tserver_master_addrs=127.8.131.62:38899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:27.440644 735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:27.442394 735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:27.455744 741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:27.455734 744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:27.456708 735 server_base.cc:1034] running on GCE node
W20260809 06:29:27.457263 742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:27.458114 735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:27.459326 735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:27.460531 735 hybrid_clock.cc:648] HybridClock initialized: now 1786256967460497 us; error 27 us; skew 500 ppm
I20260809 06:29:27.460856 735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:27 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256967, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:27.464043 735 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:29:27.465466 735 webserver.cc:460] Webserver started at http://127.8.131.3:46273/ using document root <none> and password file <none>
I20260809 06:29:27.466128 735 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:27.466293 735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:27.471745 735 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:29:27.474382 752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:27.475631 735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:29:27.475864 735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "322b47beca114cc59490353149f235c6"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "023331d978aad80e389988df55d7cea3"
server_key_iv: "3b566a00368b1ba16ebbc8c4c02ea718"
server_key_version: "encryptionkey@0"
I20260809 06:29:27.476830 735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:27.501168 735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:27.505628 735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:27.505930 735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:27.507017 735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:29:27.508409 735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:29:27.508533 735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:27.508670 735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:29:27.508762 735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:27.542282 735 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:46851
I20260809 06:29:27.542375 911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:46851 every 8 connection(s)
I20260809 06:29:27.543896 735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:29:27.545985 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 735
Aug 09 06:29:27 dist-test-slave-mjjr krb5kdc[31859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256967, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:29:27.563335 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.547148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.8.131.62:38899)
0809 06:29:27.547388 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.547396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.547809 (+ 413us) server_negotiation.cc:407] Connection header received
0809 06:29:27.548119 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.548125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.548199 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.548306 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:27.550257 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.551409 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.553475 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.553901 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.554007 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.557376 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:27.557381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:27.557384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:27.557415 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:27.560507 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.561272 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.561278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.561281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.561426 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.561848 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.561851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.561853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.562134 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:27.562431 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.562673 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.562919 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 06:29:27.568287 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.547972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:57749 (local address 127.8.131.62:38899)
0809 06:29:27.552069 (+ 4097us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.552073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.552088 (+ 15us) server_negotiation.cc:407] Connection header received
0809 06:29:27.552612 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.552616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.552716 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.552815 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:27.554683 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.555459 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.557779 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.558504 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.558646 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.562221 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:27.562227 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:27.562231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:27.562262 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:27.565575 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.566333 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.566338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.566340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.566459 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.566847 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.566850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.566852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.567085 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:27.567188 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.567879 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.568029 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4014,"thread_start_us":3837,"threads_started":1}
I20260809 06:29:27.569268 917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.548746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:57749)
0809 06:29:27.549741 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.549789 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:27.552331 (+ 2542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:27.553112 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:27.553125 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:27.553746 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:29:27.554495 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.554524 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.555665 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.555672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.557667 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.557679 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.558711 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.558716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.558967 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.559627 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:29:27.559703 (+ 76us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:29:27.562012 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:29:27.565707 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:27.565735 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:27.565751 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:27.566211 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:27.566566 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:29:27.566569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:29:27.566571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:29:27.566736 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:29:27.567192 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:29:27.567197 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:29:27.567482 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 06:29:27.567774 (+ 292us) client_negotiation.cc:240] Negotiation successful
0809 06:29:27.568125 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"spinlock_wait_cycles":22272,"thread_start_us":528,"threads_started":1}
I20260809 06:29:27.570590 912 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38899
I20260809 06:29:27.570875 912 heartbeater.cc:464] Registering TS with master...
I20260809 06:29:27.571563 912 heartbeater.cc:511] Master 127.8.131.62:38899 requested a full tablet report, sending...
I20260809 06:29:27.572902 32712 ts_manager.cc:194] Registered new tserver with Master: 322b47beca114cc59490353149f235c6 (127.8.131.3:46851)
I20260809 06:29:27.573632 32712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:57749
I20260809 06:29:27.578245 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:29:27.584671 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.579126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43010 (local address 127.8.131.62:38899)
0809 06:29:27.579278 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.579282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.579874 (+ 592us) server_negotiation.cc:407] Connection header received
0809 06:29:27.580062 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.580066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.580181 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.580296 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:29:27.580798 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.581503 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.582950 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.583347 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.583438 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.583938 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:29:27.584203 (+ 265us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 06:29:27.584472 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43010: token expired
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 06:29:27.584805 916 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43010: token expired
W20260809 06:29:27.585366 32593 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.131.62:38899)
I20260809 06:29:27.598773 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.585872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.8.131.62:38899)
0809 06:29:27.586033 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.586038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.586150 (+ 112us) server_negotiation.cc:407] Connection header received
0809 06:29:27.586322 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.586335 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.586401 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.586514 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:27.587932 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.588670 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.590047 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.590468 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.590605 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.593393 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:27.593404 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:27.593407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:27.593438 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:27.596099 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.596776 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.596782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.596786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.596930 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.597315 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.597320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.597325 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.597646 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:27.597936 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.598242 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.598461 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 06:29:27.602281 32712 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:29:27.609494 32712 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:29:27.633060 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.622125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38367 (local address 127.0.0.1:59998)
0809 06:29:27.623298 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.623320 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:27.623455 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:27.625178 (+ 1723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:27.625185 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:27.625216 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:27.625633 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.625647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.627616 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.627622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.630246 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.630262 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.632159 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.632166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.632361 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.632411 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 06:29:27.632485 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 06:29:27.632610 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"mutex_wait_us":82,"thread_start_us":907,"threads_started":1}
I20260809 06:29:27.633373 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.621686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34440 (local address 127.8.131.3:46851)
0809 06:29:27.624205 (+ 2519us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.624217 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.624253 (+ 36us) server_negotiation.cc:407] Connection header received
0809 06:29:27.624346 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.624353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.624605 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.625496 (+ 891us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:27.625792 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.627602 (+ 1810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.630093 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.631531 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.631692 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.632792 (+ 1100us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.632882 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.633072 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2286,"thread_start_us":2107,"threads_started":1}
I20260809 06:29:27.633687 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.622867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:38277 (local address 127.0.0.1:37216)
0809 06:29:27.625860 (+ 2993us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.625876 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:27.625993 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:27.626680 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:27.626686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:27.626711 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:27.627071 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.627079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.628946 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.628951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.630924 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.630935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.632813 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.632819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.633005 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.633032 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 06:29:27.633140 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 06:29:27.633261 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2881,"mutex_wait_us":36,"thread_start_us":2709,"threads_started":1}
I20260809 06:29:27.633813 933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.623516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.8.131.2:38277)
0809 06:29:27.624783 (+ 1267us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.624790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.626119 (+ 1329us) server_negotiation.cc:407] Connection header received
0809 06:29:27.626227 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.626237 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.626477 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.626696 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:27.627221 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.628757 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.631094 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.632326 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.632441 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.632622 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.633202 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.633376 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":982,"threads_started":1}
I20260809 06:29:27.634089 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.621358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:46851 (local address 127.0.0.1:34440)
0809 06:29:27.622347 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.622397 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:27.622571 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:27.624836 (+ 2265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:27.624843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:27.624882 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:27.625242 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.625261 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.627763 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.627769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.629934 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.629945 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.631784 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.631790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.632016 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.632078 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 06:29:27.632214 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 06:29:27.632433 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":22,"thread_start_us":581,"threads_started":1}
I20260809 06:29:27.634617 929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.622652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.8.131.1:38367)
0809 06:29:27.624563 (+ 1911us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.624572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.624601 (+ 29us) server_negotiation.cc:407] Connection header received
0809 06:29:27.624695 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.624706 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.624947 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.625180 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:27.625850 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.627464 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.630417 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.631928 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.633886 (+ 1958us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.634105 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.634186 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.634333 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1727,"thread_start_us":1560,"threads_started":1}
I20260809 06:29:27.637787 433 tablet_service.cc:1467] Processing CreateTablet for tablet a19793eddce4482bb8867d04749bcf23 (DEFAULT_TABLE table=test-table [id=ddec2f585a86447cbe33ba5cebd0d066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:29:27.637810 631 tablet_service.cc:1467] Processing CreateTablet for tablet a19793eddce4482bb8867d04749bcf23 (DEFAULT_TABLE table=test-table [id=ddec2f585a86447cbe33ba5cebd0d066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:29:27.637810 818 tablet_service.cc:1467] Processing CreateTablet for tablet a19793eddce4482bb8867d04749bcf23 (DEFAULT_TABLE table=test-table [id=ddec2f585a86447cbe33ba5cebd0d066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:29:27.639055 433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a19793eddce4482bb8867d04749bcf23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:29:27.639150 818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a19793eddce4482bb8867d04749bcf23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:29:27.639166 631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a19793eddce4482bb8867d04749bcf23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:29:27.650703 532 heartbeater.cc:503] Master 127.8.131.62:38899 was elected leader, sending a full tablet report...
I20260809 06:29:27.653371 934 tablet_bootstrap.cc:492] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Bootstrap starting.
I20260809 06:29:27.657191 934 tablet_bootstrap.cc:654] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Neither blocks nor log segments found. Creating new log.
I20260809 06:29:27.658044 937 tablet_bootstrap.cc:492] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: Bootstrap starting.
I20260809 06:29:27.658535 934 log.cc:824] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:29:27.660385 939 tablet_bootstrap.cc:492] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: Bootstrap starting.
I20260809 06:29:27.661934 934 tablet_bootstrap.cc:492] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: No bootstrap required, opened a new log
I20260809 06:29:27.662040 937 tablet_bootstrap.cc:654] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: Neither blocks nor log segments found. Creating new log.
I20260809 06:29:27.662223 934 ts_tablet_manager.cc:1397] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 06:29:27.663332 937 log.cc:824] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: Log is configured to *not* fsync() on all Append() calls
I20260809 06:29:27.663844 939 tablet_bootstrap.cc:654] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: Neither blocks nor log segments found. Creating new log.
I20260809 06:29:27.665788 939 log.cc:824] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: Log is configured to *not* fsync() on all Append() calls
I20260809 06:29:27.668696 939 tablet_bootstrap.cc:492] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: No bootstrap required, opened a new log
I20260809 06:29:27.668974 939 ts_tablet_manager.cc:1397] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260809 06:29:27.669180 937 tablet_bootstrap.cc:492] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: No bootstrap required, opened a new log
I20260809 06:29:27.669472 937 ts_tablet_manager.cc:1397] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260809 06:29:27.672591 934 raft_consensus.cc:348] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } }
I20260809 06:29:27.673053 934 raft_consensus.cc:374] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:29:27.673197 934 raft_consensus.cc:729] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e413d6c2cf534fdb916a2945e9e823c3, State: Initialized, Role: FOLLOWER
I20260809 06:29:27.673872 934 consensus_queue.cc:260] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } }
I20260809 06:29:27.675789 934 ts_tablet_manager.cc:1428] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Time spent starting tablet: real 0.013s user 0.004s sys 0.011s
I20260809 06:29:27.677606 939 raft_consensus.cc:348] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } }
I20260809 06:29:27.678031 939 raft_consensus.cc:374] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:29:27.678145 939 raft_consensus.cc:729] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 322b47beca114cc59490353149f235c6, State: Initialized, Role: FOLLOWER
I20260809 06:29:27.678779 939 consensus_queue.cc:260] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } }
I20260809 06:29:27.680042 912 heartbeater.cc:503] Master 127.8.131.62:38899 was elected leader, sending a full tablet report...
I20260809 06:29:27.680011 937 raft_consensus.cc:348] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } }
I20260809 06:29:27.680438 937 raft_consensus.cc:374] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:29:27.680534 937 raft_consensus.cc:729] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1517e8ba0dd6493594311e4241e59651, State: Initialized, Role: FOLLOWER
I20260809 06:29:27.680702 939 ts_tablet_manager.cc:1428] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260809 06:29:27.681347 937 consensus_queue.cc:260] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } }
I20260809 06:29:27.682516 947 raft_consensus.cc:482] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:29:27.682591 732 heartbeater.cc:503] Master 127.8.131.62:38899 was elected leader, sending a full tablet report...
I20260809 06:29:27.682840 947 raft_consensus.cc:504] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } }
I20260809 06:29:27.683192 937 ts_tablet_manager.cc:1428] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260809 06:29:27.684692 947 leader_election.cc:288] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 322b47beca114cc59490353149f235c6 (127.8.131.3:46851), 1517e8ba0dd6493594311e4241e59651 (127.8.131.2:38277)
I20260809 06:29:27.692018 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.684855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:46851 (local address 127.8.131.1:36243)
0809 06:29:27.685447 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.685469 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:27.685620 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:27.685959 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:27.685965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:27.685991 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:27.686374 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.686390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.687897 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.687906 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.690159 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.690173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.691334 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.691342 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.691483 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.691506 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 06:29:27.691578 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 06:29:27.691729 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":3,"thread_start_us":1262,"threads_started":1}
I20260809 06:29:27.692018 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.685273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:36243 (local address 127.8.131.3:46851)
0809 06:29:27.685465 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.685472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.685569 (+ 97us) server_negotiation.cc:407] Connection header received
0809 06:29:27.685775 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.685778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.685832 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.685904 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:27.686598 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.687729 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.690326 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.691089 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.691178 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.691265 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.691627 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.691775 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 06:29:27.693122 849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a19793eddce4482bb8867d04749bcf23" candidate_uuid: "e413d6c2cf534fdb916a2945e9e823c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "322b47beca114cc59490353149f235c6" is_pre_election: true
I20260809 06:29:27.693423 933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.686940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:48535 (local address 127.8.131.2:38277)
0809 06:29:27.687125 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.687131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.687191 (+ 60us) server_negotiation.cc:407] Connection header received
0809 06:29:27.687437 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.687444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.687511 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.687602 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:29:27.688384 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.689448 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.691386 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.692505 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.692638 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.692700 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.693147 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.693240 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:29:27.693651 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.686556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:38277 (local address 127.8.131.1:48535)
0809 06:29:27.687098 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.687121 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 06:29:27.687280 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:29:27.687620 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:29:27.687623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:29:27.687698 (+ 75us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:29:27.688200 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.688211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.689612 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.689617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.691181 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:29:27.691196 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.692746 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.692753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:29:27.692983 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.693024 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 06:29:27.693132 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 06:29:27.693286 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":24,"thread_start_us":574,"threads_started":1}
I20260809 06:29:27.693650 849 raft_consensus.cc:2454] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e413d6c2cf534fdb916a2945e9e823c3 in term 0.
I20260809 06:29:27.694391 365 leader_election.cc:302] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 322b47beca114cc59490353149f235c6, e413d6c2cf534fdb916a2945e9e823c3; no voters:
I20260809 06:29:27.694517 663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a19793eddce4482bb8867d04749bcf23" candidate_uuid: "e413d6c2cf534fdb916a2945e9e823c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1517e8ba0dd6493594311e4241e59651" is_pre_election: true
I20260809 06:29:27.694828 947 raft_consensus.cc:2781] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:29:27.694988 947 raft_consensus.cc:482] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:29:27.695096 947 raft_consensus.cc:3037] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:29:27.695017 663 raft_consensus.cc:2454] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e413d6c2cf534fdb916a2945e9e823c3 in term 0.
I20260809 06:29:27.697425 947 raft_consensus.cc:504] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } }
I20260809 06:29:27.698019 947 leader_election.cc:288] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [CANDIDATE]: Term 1 election: Requested vote from peers 322b47beca114cc59490353149f235c6 (127.8.131.3:46851), 1517e8ba0dd6493594311e4241e59651 (127.8.131.2:38277)
I20260809 06:29:27.698526 849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a19793eddce4482bb8867d04749bcf23" candidate_uuid: "e413d6c2cf534fdb916a2945e9e823c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "322b47beca114cc59490353149f235c6"
I20260809 06:29:27.698614 663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a19793eddce4482bb8867d04749bcf23" candidate_uuid: "e413d6c2cf534fdb916a2945e9e823c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1517e8ba0dd6493594311e4241e59651"
I20260809 06:29:27.698745 849 raft_consensus.cc:3037] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:29:27.698791 663 raft_consensus.cc:3037] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:29:27.702090 663 raft_consensus.cc:2454] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e413d6c2cf534fdb916a2945e9e823c3 in term 1.
I20260809 06:29:27.702090 849 raft_consensus.cc:2454] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e413d6c2cf534fdb916a2945e9e823c3 in term 1.
I20260809 06:29:27.702466 365 leader_election.cc:302] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1517e8ba0dd6493594311e4241e59651, e413d6c2cf534fdb916a2945e9e823c3; no voters:
I20260809 06:29:27.702775 947 raft_consensus.cc:2781] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:29:27.703230 947 raft_consensus.cc:686] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 LEADER]: Becoming Leader. State: Replica: e413d6c2cf534fdb916a2945e9e823c3, State: Running, Role: LEADER
I20260809 06:29:27.703779 947 consensus_queue.cc:237] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } }
I20260809 06:29:27.708328 32710 catalog_manager.cc:5495] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e413d6c2cf534fdb916a2945e9e823c3 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "e413d6c2cf534fdb916a2945e9e823c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e413d6c2cf534fdb916a2945e9e823c3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38367 } health_report { overall_health: HEALTHY } } }
I20260809 06:29:27.736677 929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.731106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60014 (local address 127.8.131.1:38367)
0809 06:29:27.731319 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.731325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.731348 (+ 23us) server_negotiation.cc:407] Connection header received
0809 06:29:27.731460 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.731466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.731565 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.731772 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:29:27.732154 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.732912 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.734372 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.734783 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.734927 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.735486 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:29:27.736096 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:29:27.736262 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.736321 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.736404 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:29:27.744678 663 raft_consensus.cc:1261] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [term 1 FOLLOWER]: Refusing update from remote peer e413d6c2cf534fdb916a2945e9e823c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:29:27.744743 849 raft_consensus.cc:1261] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [term 1 FOLLOWER]: Refusing update from remote peer e413d6c2cf534fdb916a2945e9e823c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:29:27.745633 952 consensus_queue.cc:1035] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1517e8ba0dd6493594311e4241e59651" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:29:27.746093 947 consensus_queue.cc:1035] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "322b47beca114cc59490353149f235c6" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:29:27.762660 956 mvcc.cc:205] Tried to move back new op lower bound from 7316508539872395264 to 7316508539718828032. Current Snapshot: MvccSnapshot[applied={T|T < 7316508539872395264}]
I20260809 06:29:27.774564 32712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:29:27.774881 32712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:29:27.778334 32712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Sending DeleteTablet for 3 replicas of tablet a19793eddce4482bb8867d04749bcf23
I20260809 06:29:27.779614 818 tablet_service.cc:1514] Processing DeleteTablet for tablet a19793eddce4482bb8867d04749bcf23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:29:27 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:34440
I20260809 06:29:27.779621 433 tablet_service.cc:1514] Processing DeleteTablet for tablet a19793eddce4482bb8867d04749bcf23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:29:27 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:59998
I20260809 06:29:27.779836 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 309
I20260809 06:29:27.780797 631 tablet_service.cc:1514] Processing DeleteTablet for tablet a19793eddce4482bb8867d04749bcf23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:29:27 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:37216
I20260809 06:29:27.790936 985 tablet_replica.cc:330] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: stopping tablet replica
I20260809 06:29:27.791745 985 raft_consensus.cc:2229] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:29:27.792711 985 raft_consensus.cc:2258] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:29:27.795756 985 ts_tablet_manager.cc:1905] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:29:27.802397 929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.781623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.8.131.1:38367)
0809 06:29:27.781796 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.781802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.781820 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:29:27.781874 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.781879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.781964 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.782085 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:27.784600 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.785944 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.787968 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.788724 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.788847 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.792873 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:27.792887 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:27.792904 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:27.792954 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:27.797665 (+ 4711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.798604 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.798614 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.798617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.798815 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.799378 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.799387 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.799391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.799939 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:27.800273 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.801709 (+ 1436us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.801970 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 06:29:27.803534 526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:27.804528 984 tablet_replica.cc:330] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: stopping tablet replica
I20260809 06:29:27.804569 986 tablet_replica.cc:330] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: stopping tablet replica
I20260809 06:29:27.805169 986 raft_consensus.cc:2229] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:29:27.805055 984 raft_consensus.cc:2229] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:29:27.805861 986 raft_consensus.cc:2258] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:29:27.805895 984 raft_consensus.cc:2258] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:29:27.808053 984 ts_tablet_manager.cc:1905] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:29:27.808234 986 ts_tablet_manager.cc:1905] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:29:27.814203 986 ts_tablet_manager.cc:1918] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:29:27.814383 986 log.cc:1196] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/a19793eddce4482bb8867d04749bcf23
I20260809 06:29:27.814844 986 ts_tablet_manager.cc:1939] T a19793eddce4482bb8867d04749bcf23 P 1517e8ba0dd6493594311e4241e59651: Deleting consensus metadata
I20260809 06:29:27.816049 984 ts_tablet_manager.cc:1918] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:29:27.816259 984 log.cc:1196] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/a19793eddce4482bb8867d04749bcf23
I20260809 06:29:27.816711 32688 catalog_manager.cc:4841] TS 1517e8ba0dd6493594311e4241e59651 (127.8.131.2:38277): tablet a19793eddce4482bb8867d04749bcf23 (table test-table [id=ddec2f585a86447cbe33ba5cebd0d066]) successfully deleted
I20260809 06:29:27.817167 984 ts_tablet_manager.cc:1939] T a19793eddce4482bb8867d04749bcf23 P 322b47beca114cc59490353149f235c6: Deleting consensus metadata
I20260809 06:29:27.819255 32688 catalog_manager.cc:4841] TS 322b47beca114cc59490353149f235c6 (127.8.131.3:46851): tablet a19793eddce4482bb8867d04749bcf23 (table test-table [id=ddec2f585a86447cbe33ba5cebd0d066]) successfully deleted
I20260809 06:29:27.910897 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 309
W20260809 06:29:27.936324 32691 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.131.1:38367 (error 108)
I20260809 06:29:27.936635 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 536
I20260809 06:29:27.938301 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.937878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38367 (local address 127.0.0.1:60030)
0809 06:29:27.938046 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.938162 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 06:29:27.938527 32691 catalog_manager.cc:4584] TS e413d6c2cf534fdb916a2945e9e823c3 (127.8.131.1:38367): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a19793eddce4482bb8867d04749bcf23: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38367: connect: Connection refused (error 111)
I20260809 06:29:27.951079 933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.937417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37230 (local address 127.8.131.2:38277)
0809 06:29:27.937623 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:27.937630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:27.937730 (+ 100us) server_negotiation.cc:407] Connection header received
0809 06:29:27.937981 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:27.937986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:27.938064 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:27.938176 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:27.939692 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.940420 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.941894 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:27.942293 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:27.942447 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:27.945384 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:27.945398 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:27.945413 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:27.945449 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:27.948444 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.949193 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.949200 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.949202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.949322 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:27.949657 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:27.949660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:27.949662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:27.950034 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:27.950244 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:27.950519 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 06:29:27.950674 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 06:29:27.951843 727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:27.992491 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.991969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38367 (local address 127.0.0.1:60032)
0809 06:29:27.992207 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.992323 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 06:29:27.993563 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:27.993186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38367 (local address 127.0.0.1:60034)
0809 06:29:27.993348 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:27.993426 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 06:29:28.042192 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 536
I20260809 06:29:28.054149 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:28.053629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38367 (local address 127.0.0.1:60036)
0809 06:29:28.053886 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:28.053979 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 06:29:28.055193 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:28.054846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38367 (local address 127.0.0.1:60046)
0809 06:29:28.055011 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:28.055078 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 06:29:28.067512 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 735
I20260809 06:29:28.082656 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:28.068431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34456 (local address 127.8.131.3:46851)
0809 06:29:28.068641 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 06:29:28.068646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:29:28.068666 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:29:28.068724 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:29:28.068728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:29:28.068793 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:29:28.068898 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:29:28.070618 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:28.071516 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:28.073080 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:29:28.073507 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:29:28.073619 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:29:28.076392 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:29:28.076407 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:29:28.076425 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:29:28.076459 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:29:28.079384 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:28.080339 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:28.080346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:28.080349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:28.080539 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:29:28.080978 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:29:28.080982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:29:28.080984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:29:28.081373 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:29:28.081648 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 06:29:28.082116 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 06:29:28.082305 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:29:28.083319 907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:28.135581 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:28.135142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38367 (local address 127.0.0.1:60056)
0809 06:29:28.135347 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:28.135433 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 06:29:28.136735 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:28.136330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38367 (local address 127.0.0.1:60072)
0809 06:29:28.136555 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:28.136624 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 06:29:28.173123 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 735
I20260809 06:29:28.196802 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 32661
I20260809 06:29:28.198004 32766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:29:28.312424 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 32661
I20260809 06:29:48.342212 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38899
--webserver_interface=127.8.131.62
--webserver_port=34311
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:38899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:29:48.759759 994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:48.760258 994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:48.760416 994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:48.772766 994 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 06:29:48.772943 994 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 06:29:48.773011 994 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 06:29:48.773097 994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:29:48.773216 994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:48.773340 994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:48.773432 994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:29:48.773519 994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:29:48.790241 994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:38899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=34311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:48.792459 994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:48.794683 994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:48.815537 1001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:48.815692 1005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:48.815533 1002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:48.817871 994 server_base.cc:1034] running on GCE node
I20260809 06:29:48.819325 994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:48.822005 994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:48.823364 994 hybrid_clock.cc:648] HybridClock initialized: now 1786256988823231 us; error 71 us; skew 500 ppm
I20260809 06:29:48.823992 994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:48 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256988, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:48.829391 994 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:29:48.832031 994 webserver.cc:460] Webserver started at http://127.8.131.62:34311/ using document root <none> and password file <none>
I20260809 06:29:48.833302 994 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:48.833500 994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:48.843060 994 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.002s
I20260809 06:29:48.847971 1014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:48.850296 994 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:29:48.850611 994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "318591c3c83e4609a9c2c9ab2a600dc5"
format_stamp: "Formatted at 2026-08-09 06:29:02 on dist-test-slave-mjjr"
server_key: "ec2cf10be60df12f0c50b508aea35f96"
server_key_iv: "49d97546cdb01ffb800d13fbf5b903df"
server_key_version: "encryptionkey@0"
I20260809 06:29:48.852385 994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:48.913594 994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:48.922060 994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:48.922662 994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:48.954583 994 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:38899
I20260809 06:29:48.954648 1094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:38899 every 8 connection(s)
I20260809 06:29:48.957804 994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:29:48.965951 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 994
I20260809 06:29:48.967779 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:38367
--local_ip_for_outbound_sockets=127.8.131.1
--tserver_master_addrs=127.8.131.62:38899
--webserver_port=36491
--webserver_interface=127.8.131.1
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:29:48.969978 1095 sys_catalog.cc:263] Verifying existing consensus state
I20260809 06:29:48.976274 1095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Bootstrap starting.
I20260809 06:29:49.014477 1095 log.cc:824] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Log is configured to *not* fsync() on all Append() calls
I20260809 06:29:49.045358 1095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 06:29:49.046520 1095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Bootstrap complete.
I20260809 06:29:49.063422 1095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:49.064813 1095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 318591c3c83e4609a9c2c9ab2a600dc5, State: Initialized, Role: FOLLOWER
I20260809 06:29:49.065761 1095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:49.066115 1095 raft_consensus.cc:388] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:29:49.066293 1095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:29:49.066566 1095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 06:29:49.073581 1095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:49.074378 1095 leader_election.cc:302] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 318591c3c83e4609a9c2c9ab2a600dc5; no voters:
I20260809 06:29:49.075531 1095 leader_election.cc:288] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 06:29:49.075865 1100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 06:29:49.077943 1100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [term 3 LEADER]: Becoming Leader. State: Replica: 318591c3c83e4609a9c2c9ab2a600dc5, State: Running, Role: LEADER
I20260809 06:29:49.078656 1100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } }
I20260809 06:29:49.079058 1095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:29:49.081862 1101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } } }
I20260809 06:29:49.082293 1101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 06:29:49.082950 1103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 318591c3c83e4609a9c2c9ab2a600dc5. Latest consensus state: current_term: 3 leader_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318591c3c83e4609a9c2c9ab2a600dc5" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38899 } } }
I20260809 06:29:49.083686 1103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 06:29:49.086054 1110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:29:49.097332 1110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:29:49.099413 1110 catalog_manager.cc:1259] Loaded cluster ID: ef21f7a911a848dfb9ec29837a4bd4eb
I20260809 06:29:49.099615 1110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:29:49.105278 1110 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:29:49.107733 1110 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Loaded TSK: 0
I20260809 06:29:49.108000 1110 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 318591c3c83e4609a9c2c9ab2a600dc5: Loaded TSK: 1
F20260809 06:29:49.109797 1110 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f25e94399ad google::LogMessage::Fail() at ??:0
@ 0x7f25e943c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f25e94394de google::LogMessage::Flush() at ??:0
@ 0x7f25e943aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f25ed19d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f25ed18c73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f25fee681c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f25fee67783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f25feef32a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f25feef30c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f25fef3d5e9 std::function<>::operator()() at ??:0
@ 0x7f25fee6bcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f25fee69c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f25feef1610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f25feef1439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f25ec085345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f25ec08891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f25ec088799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f25ec063eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f25ef33d6db start_thread at ??:0
@ 0x7f25e82d361f clone at ??:0
W20260809 06:29:49.422710 1098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:49.423015 1098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:49.423177 1098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:49.430361 1098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:49.430501 1098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:49.430677 1098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:29:49.440080 1098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:38367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=36491
--tserver_master_addrs=127.8.131.62:38899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:49.441255 1098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:49.442531 1098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:49.455111 1169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:49.455200 1167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:49.455173 1165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:49.456068 1098 server_base.cc:1034] running on GCE node
I20260809 06:29:49.456667 1098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:49.457523 1098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:49.458741 1098 hybrid_clock.cc:648] HybridClock initialized: now 1786256989458704 us; error 46 us; skew 500 ppm
I20260809 06:29:49.459051 1098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:49 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256989, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:49.462097 1098 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:29:49.463514 1098 webserver.cc:460] Webserver started at http://127.8.131.1:36491/ using document root <none> and password file <none>
I20260809 06:29:49.464221 1098 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:49.464363 1098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:49.469441 1098 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 06:29:49.472260 1180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:49.473539 1098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:29:49.473694 1098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "e413d6c2cf534fdb916a2945e9e823c3"
format_stamp: "Formatted at 2026-08-09 06:29:03 on dist-test-slave-mjjr"
server_key: "69b99fe9f22a8469e91f107e8b34d670"
server_key_iv: "81561093bee22c02ea8f10d3d66ab398"
server_key_version: "encryptionkey@0"
I20260809 06:29:49.474612 1098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:49.487783 1098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:49.491422 1098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:49.491741 1098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:49.492722 1098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:29:49.494520 1188 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260809 06:29:49.499596 1188 ts_tablet_manager.cc:1852] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 06:29:49.500669 1188 ts_tablet_manager.cc:1905] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:29:49.507486 1188 ts_tablet_manager.cc:1918] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:29:49.507702 1188 log.cc:1196] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/a19793eddce4482bb8867d04749bcf23
I20260809 06:29:49.508178 1188 ts_tablet_manager.cc:1939] T a19793eddce4482bb8867d04749bcf23 P e413d6c2cf534fdb916a2945e9e823c3: Deleting consensus metadata
I20260809 06:29:49.508490 1098 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 06:29:49.508625 1098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.014s user 0.001s sys 0.000s
I20260809 06:29:49.508751 1098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:29:49.508847 1098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:49.544178 1098 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:38367
I20260809 06:29:49.544262 1342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:38367 every 8 connection(s)
I20260809 06:29:49.545537 1098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:29:49.550271 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:49.548467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:60635)
0809 06:29:49.549210 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:49.549442 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":260,"threads_started":1}
W20260809 06:29:49.550905 1345 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:38899 (0 consecutive failures): Network error: Failed to ping master at 127.8.131.62:38899: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
I20260809 06:29:49.551729 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:49.551366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:46787)
0809 06:29:49.551513 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:49.551583 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 06:29:49.552510 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:49.552210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:60705)
0809 06:29:49.552325 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:49.552394 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 06:29:49.552734 1345 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 06:29:49.555038 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 1098
I20260809 06:29:49.555718 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:38277
--local_ip_for_outbound_sockets=127.8.131.2
--tserver_master_addrs=127.8.131.62:38899
--webserver_port=45399
--webserver_interface=127.8.131.2
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:29:49.895328 1349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:49.895587 1349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:49.895771 1349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:49.903033 1349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:49.903126 1349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:49.903288 1349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:29:49.912514 1349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:38277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=45399
--tserver_master_addrs=127.8.131.62:38899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:49.913686 1349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:49.915185 1349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:49.928244 1357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:49.928262 1358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:49.928488 1361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:49.929203 1349 server_base.cc:1034] running on GCE node
I20260809 06:29:49.929821 1349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:49.930753 1349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:49.931986 1349 hybrid_clock.cc:648] HybridClock initialized: now 1786256989931936 us; error 38 us; skew 500 ppm
I20260809 06:29:49.932294 1349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:49 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256989, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:49.935431 1349 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:29:49.936924 1349 webserver.cc:460] Webserver started at http://127.8.131.2:45399/ using document root <none> and password file <none>
I20260809 06:29:49.937631 1349 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:49.937777 1349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:49.943212 1349 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260809 06:29:49.945938 1369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:49.947254 1349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:29:49.947432 1349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "1517e8ba0dd6493594311e4241e59651"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "4a0c149c988f61e90aec1822d3459a83"
server_key_iv: "8f1efa949de3cf1a61195b375cbdaf48"
server_key_version: "encryptionkey@0"
I20260809 06:29:49.948472 1349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:49.960126 1349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:49.964084 1349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:49.964350 1349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:49.965282 1349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:29:49.966588 1349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:29:49.966708 1349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:49.966835 1349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:29:49.966923 1349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:50.001279 1349 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:38277
I20260809 06:29:50.001371 1526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:38277 every 8 connection(s)
I20260809 06:29:50.002761 1349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:29:50.003486 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 1349
I20260809 06:29:50.004333 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:46851
--local_ip_for_outbound_sockets=127.8.131.3
--tserver_master_addrs=127.8.131.62:38899
--webserver_port=46273
--webserver_interface=127.8.131.3
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:29:50.008157 1531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:50.006208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:43323)
0809 06:29:50.007006 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:50.007202 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":326,"threads_started":1}
W20260809 06:29:50.008922 1527 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:38899 (0 consecutive failures): Network error: Failed to ping master at 127.8.131.62:38899: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
I20260809 06:29:50.009784 1531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:50.009479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:40615)
0809 06:29:50.009615 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:50.009680 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 06:29:50.010655 1531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:50.010298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:40331)
0809 06:29:50.010433 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:50.010556 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 06:29:50.010900 1527 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 06:29:50.341521 1532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:29:50.341797 1532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:29:50.341898 1532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:29:50.348938 1532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:29:50.349035 1532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:29:50.349149 1532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:29:50.358619 1532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:46851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=46273
--tserver_master_addrs=127.8.131.62:38899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:29:50.360170 1532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:29:50.361456 1532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:29:50.374684 1544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:50.374800 1540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:29:50.374837 1541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:29:50.375669 1532 server_base.cc:1034] running on GCE node
I20260809 06:29:50.376284 1532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:29:50.377131 1532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:29:50.378394 1532 hybrid_clock.cc:648] HybridClock initialized: now 1786256990378325 us; error 51 us; skew 500 ppm
I20260809 06:29:50.378649 1532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:29:50 dist-test-slave-mjjr krb5kdc[31859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786256990, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:29:50.386598 1532 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:29:50.388154 1532 webserver.cc:460] Webserver started at http://127.8.131.3:46273/ using document root <none> and password file <none>
I20260809 06:29:50.388829 1532 fs_manager.cc:359] Metadata directory not provided
I20260809 06:29:50.388970 1532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:29:50.394363 1532 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 06:29:50.397183 1552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:50.398506 1532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:29:50.398684 1532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "322b47beca114cc59490353149f235c6"
format_stamp: "Formatted at 2026-08-09 06:29:04 on dist-test-slave-mjjr"
server_key: "023331d978aad80e389988df55d7cea3"
server_key_iv: "3b566a00368b1ba16ebbc8c4c02ea718"
server_key_version: "encryptionkey@0"
I20260809 06:29:50.399642 1532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:29:50.420037 1532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:29:50.423928 1532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:29:50.424202 1532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:29:50.425208 1532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:29:50.426483 1532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:29:50.426569 1532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:50.426652 1532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:29:50.426708 1532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:29:50.460227 1532 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:46851
I20260809 06:29:50.460295 1708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:46851 every 8 connection(s)
I20260809 06:29:50.461634 1532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:29:50.465402 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 1532
I20260809 06:29:50.466773 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:50.464704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:55345)
0809 06:29:50.465595 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:50.465853 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":321,"threads_started":1}
W20260809 06:29:50.467506 1709 heartbeater.cc:644] Failed to heartbeat to 127.8.131.62:38899 (0 consecutive failures): Network error: Failed to ping master at 127.8.131.62:38899: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
I20260809 06:29:50.468276 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:50.467987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:47737)
0809 06:29:50.468111 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:50.468181 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 06:29:50.469027 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:50.468736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:45147)
0809 06:29:50.468849 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:50.468917 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 06:29:50.469259 1709 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 06:29:50.553977 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:50.553206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:44355)
0809 06:29:50.553675 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:50.553760 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":237,"threads_started":1}
I20260809 06:29:51.012244 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:51.011423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:33323)
0809 06:29:51.011911 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:51.012021 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":258,"threads_started":1}
I20260809 06:29:51.470510 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:51.469729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:37123)
0809 06:29:51.470195 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:51.470289 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":235,"threads_started":1}
I20260809 06:29:51.555554 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:51.554798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:60341)
0809 06:29:51.555316 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:51.555394 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":283,"threads_started":1}
I20260809 06:29:52.013909 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:52.013165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:55843)
0809 06:29:52.013652 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:52.013729 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":235,"threads_started":1}
I20260809 06:29:52.472189 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:52.471415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:50425)
0809 06:29:52.471938 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:52.472019 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":283,"threads_started":1}
I20260809 06:29:52.557094 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:52.556367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:39523)
0809 06:29:52.556841 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:52.556925 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":241,"threads_started":1}
I20260809 06:29:53.015432 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:53.014738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:60465)
0809 06:29:53.015190 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:53.015268 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":245,"threads_started":1}
I20260809 06:29:53.473686 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:53.473011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:47397)
0809 06:29:53.473444 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:53.473527 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":229,"threads_started":1}
I20260809 06:29:53.558918 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:53.558046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:44541)
0809 06:29:53.558599 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:53.558707 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":262,"threads_started":1}
I20260809 06:29:54.017066 1731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:54.016330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:57009)
0809 06:29:54.016781 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:54.016859 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":231,"threads_started":1}
I20260809 06:29:54.475329 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:54.474577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:55495)
0809 06:29:54.475051 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:54.475130 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":249,"threads_started":1}
I20260809 06:29:54.560674 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:54.559910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:53777)
0809 06:29:54.560414 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:54.560497 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":273,"threads_started":1}
I20260809 06:29:55.018620 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:55.017944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:47257)
0809 06:29:55.018377 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:55.018455 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":220,"threads_started":1}
I20260809 06:29:55.476994 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:55.476295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:52429)
0809 06:29:55.476730 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:55.476808 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":220,"threads_started":1}
I20260809 06:29:55.562211 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:55.561530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:53631)
0809 06:29:55.561969 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:55.562049 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":235,"threads_started":1}
I20260809 06:29:56.020275 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:56.019493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:33179)
0809 06:29:56.020009 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:56.020100 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":275,"threads_started":1}
I20260809 06:29:56.478724 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:56.477850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:52169)
0809 06:29:56.478444 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:56.478560 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":358,"threads_started":1}
I20260809 06:29:56.564105 1743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:56.563114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:51437)
0809 06:29:56.563776 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:56.563872 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":276,"threads_started":1}
I20260809 06:29:57.022056 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:57.021234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:38517)
0809 06:29:57.021736 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:57.021835 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":249,"threads_started":1}
I20260809 06:29:57.480672 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:57.479717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:35371)
0809 06:29:57.480338 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:57.480439 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":299,"threads_started":1}
I20260809 06:29:57.565711 1746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:57.565007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:50833)
0809 06:29:57.565455 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:57.565536 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":236,"threads_started":1}
I20260809 06:29:58.023779 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:58.022991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:34859)
0809 06:29:58.023460 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:58.023543 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":230,"threads_started":1}
I20260809 06:29:58.482468 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:58.481545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:60537)
0809 06:29:58.482072 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:58.482185 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":221,"threads_started":1}
I20260809 06:29:58.568181 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:58.566653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:56993)
0809 06:29:58.567822 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:58.567997 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":949,"thread_start_us":788,"threads_started":1}
I20260809 06:29:59.025519 1751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:59.024731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:47613)
0809 06:29:59.025258 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:59.025344 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":265,"threads_started":1}
I20260809 06:29:59.484175 1752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:59.483363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:40961)
0809 06:29:59.483923 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:59.484000 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":320,"threads_started":1}
I20260809 06:29:59.570065 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:29:59.569189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:52919)
0809 06:29:59.569711 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 06:29:59.569821 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":309,"threads_started":1}
I20260809 06:30:00.027472 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:00.026666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:48261)
0809 06:30:00.027219 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:00.027302 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":286,"threads_started":1}
I20260809 06:30:00.485751 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:00.485056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:40587)
0809 06:30:00.485509 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:00.485593 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":242,"threads_started":1}
I20260809 06:30:00.572458 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:00.571496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:45365)
0809 06:30:00.572086 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:00.572205 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":356,"threads_started":1}
I20260809 06:30:01.029162 1759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:01.028404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:48019)
0809 06:30:01.028877 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:01.028976 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":246,"threads_started":1}
I20260809 06:30:01.487628 1761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:01.486669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:46615)
0809 06:30:01.487350 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:01.487441 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":447,"threads_started":1}
I20260809 06:30:01.574141 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:01.573400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:55309)
0809 06:30:01.573849 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:01.573950 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":239,"threads_started":1}
I20260809 06:30:02.030890 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:02.030199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:52517)
0809 06:30:02.030632 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:02.030731 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":213,"threads_started":1}
I20260809 06:30:02.489440 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:02.488621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:52383)
0809 06:30:02.489110 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:02.489227 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":245,"threads_started":1}
I20260809 06:30:02.575982 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:02.575142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:41069)
0809 06:30:02.575611 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:02.575778 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":231,"threads_started":1}
I20260809 06:30:03.032716 1767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:03.031932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:57205)
0809 06:30:03.032431 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:03.032537 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":235,"threads_started":1}
I20260809 06:30:03.491170 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:03.490436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:52919)
0809 06:30:03.490900 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:03.490988 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":239,"threads_started":1}
I20260809 06:30:03.577636 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:03.576858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:58595)
0809 06:30:03.577299 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:03.577380 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
I20260809 06:30:04.034379 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:04.033674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:58965)
0809 06:30:04.034116 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:04.034198 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":224,"threads_started":1}
I20260809 06:30:04.493144 1772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:04.492242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:50683)
0809 06:30:04.492791 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:04.492906 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":316,"threads_started":1}
I20260809 06:30:04.579798 1778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:04.578851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:56197)
0809 06:30:04.579415 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:04.579516 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":247,"threads_started":1}
I20260809 06:30:05.036268 1781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:05.035491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.2:36371)
0809 06:30:05.035967 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:05.036078 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":267,"threads_started":1}
I20260809 06:30:05.476223 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 1098
I20260809 06:30:05.494980 1783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:05.494156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.3:58589)
0809 06:30:05.494672 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:05.494800 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":293,"threads_started":1}
I20260809 06:30:05.495126 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:05.477407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.8.131.1:38367)
0809 06:30:05.478314 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:05.478335 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:05.478395 (+ 60us) server_negotiation.cc:407] Connection header received
0809 06:30:05.480435 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:05.480446 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:05.480868 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:05.481182 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:05.482618 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:05.483996 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:05.485231 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:05.485851 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:05.485999 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:05.488679 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:05.488687 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:05.488700 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:05.488741 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:05.491500 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:05.492255 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:05.492263 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:05.492294 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:05.492551 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:05.493009 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:05.493013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:05.493015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:05.493531 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:05.493881 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:05.494361 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 06:30:05.494800 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":729,"threads_started":1}
I20260809 06:30:05.496611 1338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:05.584473 1786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:05.582894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38899 (local address 127.8.131.1:45553)
0809 06:30:05.584053 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:05.584199 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.62:38899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1047,"thread_start_us":915,"threads_started":1}
I20260809 06:30:05.587836 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 1098
I20260809 06:30:05.615242 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 1349
I20260809 06:30:05.633433 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:05.616282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52060 (local address 127.8.131.2:38277)
0809 06:30:05.616826 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:05.616840 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:05.616879 (+ 39us) server_negotiation.cc:407] Connection header received
0809 06:30:05.618550 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:05.618559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:05.619002 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:05.619371 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:05.621013 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:05.622217 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:05.623517 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:05.624289 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:05.624499 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:05.627181 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:05.627189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:05.627203 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:05.627245 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:05.630008 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:05.630709 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:05.630724 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:05.630732 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:05.630901 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:05.631311 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:05.631316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:05.631319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:05.631907 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:05.632246 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:05.632719 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 06:30:05.633144 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":255,"threads_started":1}
I20260809 06:30:05.634838 1521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:05.720275 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 1349
I20260809 06:30:05.744258 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 1532
I20260809 06:30:05.762168 1790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:05.745623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.8.131.3:46851)
0809 06:30:05.746268 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:05.746284 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:05.746349 (+ 65us) server_negotiation.cc:407] Connection header received
0809 06:30:05.748079 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:05.748089 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:05.748506 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:05.748794 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:05.750134 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:05.751316 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:05.752860 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:05.753509 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:05.753654 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:05.756359 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:05.756367 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:05.756382 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:05.756420 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:05.759176 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:05.759910 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:05.759916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:05.759922 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:05.760098 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:05.760460 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:05.760463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:05.760465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:05.760969 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:05.761199 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:05.761529 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 06:30:05.761864 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":313,"threads_started":1}
I20260809 06:30:05.763514 1703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:05.847829 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 1532
2026-08-09T06:30:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64940 ms)
[----------] 30 tests from SecurityITest (306147 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:05 dist-test-slave-mjjr krb5kdc[1794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:05 dist-test-slave-mjjr krb5kdc[1794](info): set up 2 sockets
Aug 09 06:30:05 dist-test-slave-mjjr krb5kdc[1794](info): commencing operation
krb5kdc: starting...
W20260809 06:30:07.171128 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.008s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:07 dist-test-slave-mjjr krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:30:07.453433 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:36939
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:36939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:07.794863 1814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:07.795147 1814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:07.795239 1814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:07.802502 1814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:07.802605 1814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:07.802659 1814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:07.802717 1814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:07.802762 1814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:07.812604 1814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:36939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:36939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:07.813864 1814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:07.815122 1814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:07.827226 1822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:07.827417 1821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:07.827672 1814 server_base.cc:1034] running on GCE node
W20260809 06:30:07.827891 1824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:07.828720 1814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:07.830096 1814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:07.831295 1814 hybrid_clock.cc:648] HybridClock initialized: now 1786257007831261 us; error 31 us; skew 500 ppm
I20260809 06:30:07.831599 1814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:07 dist-test-slave-mjjr krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257007, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:07.834718 1814 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:30:07.836143 1814 webserver.cc:460] Webserver started at http://127.8.131.62:33075/ using document root <none> and password file <none>
I20260809 06:30:07.836784 1814 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:07.836885 1814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:07.837138 1814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:07.840082 1814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98"
format_stamp: "Formatted at 2026-08-09 06:30:07 on dist-test-slave-mjjr"
server_key: "70b196cc468c554419c7403e9c297c70"
server_key_iv: "4917a0d149d3e27dfe50d4c1e9250b7c"
server_key_version: "encryptionkey@0"
I20260809 06:30:07.840790 1814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98"
format_stamp: "Formatted at 2026-08-09 06:30:07 on dist-test-slave-mjjr"
server_key: "70b196cc468c554419c7403e9c297c70"
server_key_iv: "4917a0d149d3e27dfe50d4c1e9250b7c"
server_key_version: "encryptionkey@0"
I20260809 06:30:07.845835 1814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:30:07.849150 1835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:07.850402 1814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:30:07.850589 1814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98"
format_stamp: "Formatted at 2026-08-09 06:30:07 on dist-test-slave-mjjr"
server_key: "70b196cc468c554419c7403e9c297c70"
server_key_iv: "4917a0d149d3e27dfe50d4c1e9250b7c"
server_key_version: "encryptionkey@0"
I20260809 06:30:07.850780 1814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:07.889520 1814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:07.893729 1814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:07.894078 1814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:07.913520 1814 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:36939
I20260809 06:30:07.913538 1908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:36939 every 8 connection(s)
I20260809 06:30:07.915184 1814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:07.919263 1909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:07.925208 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 1814
I20260809 06:30:07.925458 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:07.926137 8716 external_mini_cluster.cc:1428] Setting key 5a9bbce66ca67f6e33ed6a14b603565a
I20260809 06:30:07.930831 1909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98: Bootstrap starting.
I20260809 06:30:07.934957 1909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:07.936321 1909 log.cc:824] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:07.939756 1909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98: No bootstrap required, opened a new log
Aug 09 06:30:07 dist-test-slave-mjjr krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:07.949719 1909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36939 } }
I20260809 06:30:07.950196 1909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:07.950317 1909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abbbd5fd9824a7b95b9ac6a1251fc98, State: Initialized, Role: FOLLOWER
I20260809 06:30:07.951061 1909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36939 } }
I20260809 06:30:07.951395 1909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:07.951568 1909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:07.951812 1909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:07.953040 1915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:07.928441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33970 (local address 127.8.131.62:36939)
0809 06:30:07.929284 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:07.929303 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:07.929358 (+ 55us) server_negotiation.cc:407] Connection header received
0809 06:30:07.931879 (+ 2521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:07.931906 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:07.932395 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:07.932880 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:07.934275 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:07.936440 (+ 2165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:07.938267 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:07.939331 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:07.939517 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:07.943906 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:07.943920 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:07.943942 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:07.944003 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:07.948237 (+ 4234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:07.948938 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:07.948947 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:07.948964 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:07.949173 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:07.949534 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:07.949539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:07.949542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:07.950269 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:07.950593 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:07.951058 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 06:30:07.951538 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":376,"threads_started":1}
I20260809 06:30:07.955258 1909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36939 } }
I20260809 06:30:07.955886 1909 leader_election.cc:302] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8abbbd5fd9824a7b95b9ac6a1251fc98; no voters:
I20260809 06:30:07.956576 1909 leader_election.cc:288] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:07.956709 1917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:07.957450 1917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [term 1 LEADER]: Becoming Leader. State: Replica: 8abbbd5fd9824a7b95b9ac6a1251fc98, State: Running, Role: LEADER
I20260809 06:30:07.958063 1917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36939 } }
I20260809 06:30:07.958735 1909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:07.961241 1921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8abbbd5fd9824a7b95b9ac6a1251fc98. Latest consensus state: current_term: 1 leader_uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36939 } } }
I20260809 06:30:07.961598 1921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:07.962057 1920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abbbd5fd9824a7b95b9ac6a1251fc98" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 36939 } } }
I20260809 06:30:07.962370 1920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:07.963675 1930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:07.969933 1930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:07.978513 1930 catalog_manager.cc:1347] Generated new cluster ID: 633790d15331438490d311084ede775c
I20260809 06:30:07.978631 1930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:08.001379 1930 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:08.002750 1930 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:08.018253 1930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98: Generated new TSK 0
I20260809 06:30:08.019397 1930 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:30:08.099057 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36939
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:08.441622 1953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:08.441896 1953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:08.442062 1953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:08.449450 1953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:08.449584 1953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:08.449772 1953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:30:08.459988 1953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:08.461290 1953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:08.462587 1953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:08.475934 1963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:08.476063 1961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:08.476065 1960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:08.477137 1953 server_base.cc:1034] running on GCE node
I20260809 06:30:08.477746 1953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:08.478695 1953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:08.479926 1953 hybrid_clock.cc:648] HybridClock initialized: now 1786257008479890 us; error 80 us; skew 500 ppm
I20260809 06:30:08.480245 1953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:08 dist-test-slave-mjjr krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257008, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:08.483351 1953 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:30:08.484858 1953 webserver.cc:460] Webserver started at http://127.8.131.1:43949/ using document root <none> and password file <none>
I20260809 06:30:08.485562 1953 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:08.485713 1953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:08.486037 1953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:08.489074 1953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "a5be3f2f542540b08b3f309f34d35f59"
format_stamp: "Formatted at 2026-08-09 06:30:08 on dist-test-slave-mjjr"
server_key: "d0997568e519061154e093b82b86fbff"
server_key_iv: "a71a156a710774432f100db4f6c69c56"
server_key_version: "encryptionkey@0"
I20260809 06:30:08.489869 1953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "a5be3f2f542540b08b3f309f34d35f59"
format_stamp: "Formatted at 2026-08-09 06:30:08 on dist-test-slave-mjjr"
server_key: "d0997568e519061154e093b82b86fbff"
server_key_iv: "a71a156a710774432f100db4f6c69c56"
server_key_version: "encryptionkey@0"
I20260809 06:30:08.494985 1953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 06:30:08.498482 1971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:08.499763 1953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 06:30:08.499933 1953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "a5be3f2f542540b08b3f309f34d35f59"
format_stamp: "Formatted at 2026-08-09 06:30:08 on dist-test-slave-mjjr"
server_key: "d0997568e519061154e093b82b86fbff"
server_key_iv: "a71a156a710774432f100db4f6c69c56"
server_key_version: "encryptionkey@0"
I20260809 06:30:08.500140 1953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:08.528249 1953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:08.532382 1953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:08.532723 1953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:08.533815 1953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:08.535135 1953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:08.535240 1953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:08.535395 1953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:08.535480 1953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:08.569276 1953 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:45253
I20260809 06:30:08.569331 2127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:45253 every 8 connection(s)
I20260809 06:30:08.570637 1953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:30:08.578045 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 1953
I20260809 06:30:08.578301 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:30:08.578985 8716 external_mini_cluster.cc:1428] Setting key fab35f42cf332c3b7ecab99201acd1d5
Aug 09 06:30:08 dist-test-slave-mjjr krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257008, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:08.590735 2134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:08.573295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:35729 (local address 127.8.131.62:36939)
0809 06:30:08.573693 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:08.573697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:08.574694 (+ 997us) server_negotiation.cc:407] Connection header received
0809 06:30:08.576534 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:08.576538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:08.576608 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:08.576708 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:08.578461 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:08.579241 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:08.580928 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:08.581323 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:08.581460 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:08.584924 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:08.584928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:08.584932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:08.584961 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:08.587691 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:08.588414 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:08.588420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:08.588423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:08.588550 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:08.588989 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:08.588994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:08.588997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:08.589261 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:08.589428 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:08.590138 (+ 710us) server_negotiation.cc:299] Negotiation successful
0809 06:30:08.590380 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":188,"threads_started":1}
I20260809 06:30:08.591718 2135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:08.573827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:36939 (local address 127.8.131.1:35729)
0809 06:30:08.574515 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:08.574552 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:08.576269 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:08.576901 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:08.576908 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:08.577410 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:08.578230 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:08.578258 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:08.579410 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:08.579415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:08.580769 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:08.580781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:08.581550 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:08.581558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:08.581797 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:08.582470 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:30:08.582492 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:08.584702 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:08.587840 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:08.587848 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:08.587864 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:08.588284 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:08.588661 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:08.588666 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:08.588670 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:08.588867 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:08.589405 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:08.589412 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:08.589749 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 06:30:08.590017 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 06:30:08.590537 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":280,"threads_started":1}
I20260809 06:30:08.593271 2129 heartbeater.cc:347] Connected to a master server at 127.8.131.62:36939
I20260809 06:30:08.593580 2129 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:08.594275 2129 heartbeater.cc:511] Master 127.8.131.62:36939 requested a full tablet report, sending...
I20260809 06:30:08.596752 1862 ts_manager.cc:194] Registered new tserver with Master: a5be3f2f542540b08b3f309f34d35f59 (127.8.131.1:45253)
I20260809 06:30:08.598240 1862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:35729
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:30:08.644459 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36939
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:08.989308 2139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:08.989565 2139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:08.989658 2139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:08.996577 2139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:08.996682 2139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:08.996793 2139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:30:09.006481 2139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:09.007609 2139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:09.008855 2139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:09.021600 2146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:09.021680 2147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:09.021704 2149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:09.022575 2139 server_base.cc:1034] running on GCE node
I20260809 06:30:09.023289 2139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:09.024291 2139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:09.025525 2139 hybrid_clock.cc:648] HybridClock initialized: now 1786257009025453 us; error 35 us; skew 500 ppm
I20260809 06:30:09.025961 2139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:09 dist-test-slave-mjjr krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257009, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:09.029127 2139 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:30:09.030539 2139 webserver.cc:460] Webserver started at http://127.8.131.2:44115/ using document root <none> and password file <none>
I20260809 06:30:09.031208 2139 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:09.031359 2139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:09.031755 2139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:09.034622 2139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "8c562d6c8b894fdbb76de657501871f0"
format_stamp: "Formatted at 2026-08-09 06:30:09 on dist-test-slave-mjjr"
server_key: "b3e1f0ea9528e02216b8676097f67ec8"
server_key_iv: "e574759bffcb9cbe08493c383697f72f"
server_key_version: "encryptionkey@0"
I20260809 06:30:09.035415 2139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "8c562d6c8b894fdbb76de657501871f0"
format_stamp: "Formatted at 2026-08-09 06:30:09 on dist-test-slave-mjjr"
server_key: "b3e1f0ea9528e02216b8676097f67ec8"
server_key_iv: "e574759bffcb9cbe08493c383697f72f"
server_key_version: "encryptionkey@0"
I20260809 06:30:09.040750 2139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 06:30:09.044068 2157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:09.045337 2139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 06:30:09.045504 2139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "8c562d6c8b894fdbb76de657501871f0"
format_stamp: "Formatted at 2026-08-09 06:30:09 on dist-test-slave-mjjr"
server_key: "b3e1f0ea9528e02216b8676097f67ec8"
server_key_iv: "e574759bffcb9cbe08493c383697f72f"
server_key_version: "encryptionkey@0"
I20260809 06:30:09.045701 2139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:09.056782 2139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:09.060684 2139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:09.060976 2139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:09.061894 2139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:09.063205 2139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:09.063318 2139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:09.063488 2139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:09.063560 2139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:09.098246 2139 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:45519
I20260809 06:30:09.098305 2320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:45519 every 8 connection(s)
I20260809 06:30:09.099593 2139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:30:09.102651 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 2139
I20260809 06:30:09.102861 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:30:09.103368 8716 external_mini_cluster.cc:1428] Setting key 99cbdac0bf02ca083c924d4abddc54e2
Aug 09 06:30:09 dist-test-slave-mjjr krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257009, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:09.123600 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.102213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:46941 (local address 127.8.131.62:36939)
0809 06:30:09.102780 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:09.102789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:09.103918 (+ 1129us) server_negotiation.cc:407] Connection header received
0809 06:30:09.106509 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:09.106517 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:09.106628 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:09.106762 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:09.109241 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.109993 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.112662 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.113111 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.113208 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.117665 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:09.117670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:09.117673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:09.117704 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:09.120542 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:09.121291 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:09.121295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:09.121297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:09.121424 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:09.121874 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:09.121885 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:09.121887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:09.122146 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:09.122320 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:09.122984 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 06:30:09.123206 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":257,"threads_started":1}
I20260809 06:30:09.124547 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.102785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:36939 (local address 127.8.131.2:46941)
0809 06:30:09.103726 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:09.103787 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:09.106129 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:09.107220 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:09.107234 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:09.108142 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:09.109044 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.109072 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.110161 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.110172 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.112507 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.112528 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.113346 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.113356 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.113711 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.114477 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:30:09.114515 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:09.117400 (+ 2885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:09.120684 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:09.120693 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:09.120708 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:09.121128 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:09.121546 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:09.121551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:09.121555 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:09.121745 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:09.122279 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:09.122286 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:09.122618 (+ 332us) client_negotiation.cc:769] Sending connection context
0809 06:30:09.122875 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 06:30:09.123379 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":356,"threads_started":1}
I20260809 06:30:09.126046 2321 heartbeater.cc:347] Connected to a master server at 127.8.131.62:36939
I20260809 06:30:09.126416 2321 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:09.127401 2321 heartbeater.cc:511] Master 127.8.131.62:36939 requested a full tablet report, sending...
I20260809 06:30:09.129232 1862 ts_manager.cc:194] Registered new tserver with Master: 8c562d6c8b894fdbb76de657501871f0 (127.8.131.2:45519)
I20260809 06:30:09.130648 1862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:46941
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:30:09.174856 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36939
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:09.511286 2327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:09.511593 2327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:09.511785 2327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:09.518699 2327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:09.518837 2327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:09.519012 2327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:30:09.528437 2327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:36939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:09.529621 2327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:09.530901 2327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:09.544620 2332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:09.544535 2333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:09.544588 2335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:09.545647 2327 server_base.cc:1034] running on GCE node
I20260809 06:30:09.546232 2327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:09.547117 2327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:09.548317 2327 hybrid_clock.cc:648] HybridClock initialized: now 1786257009548247 us; error 55 us; skew 500 ppm
I20260809 06:30:09.548621 2327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:09 dist-test-slave-mjjr krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257009, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:09.551959 2327 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:30:09.553372 2327 webserver.cc:460] Webserver started at http://127.8.131.3:35993/ using document root <none> and password file <none>
I20260809 06:30:09.554097 2327 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:09.554255 2327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:09.554590 2327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:09.557600 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "bd73138796e6438c905eb4f1888895ba"
format_stamp: "Formatted at 2026-08-09 06:30:09 on dist-test-slave-mjjr"
server_key: "2fe6c06b2de5c00eac3943b6aa54bce9"
server_key_iv: "ed9bb0ec2a557c4d2e4dd64945e1b5b7"
server_key_version: "encryptionkey@0"
I20260809 06:30:09.558462 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "bd73138796e6438c905eb4f1888895ba"
format_stamp: "Formatted at 2026-08-09 06:30:09 on dist-test-slave-mjjr"
server_key: "2fe6c06b2de5c00eac3943b6aa54bce9"
server_key_iv: "ed9bb0ec2a557c4d2e4dd64945e1b5b7"
server_key_version: "encryptionkey@0"
I20260809 06:30:09.563958 2327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:30:09.567517 2345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:09.569054 2327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:09.569216 2327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "bd73138796e6438c905eb4f1888895ba"
format_stamp: "Formatted at 2026-08-09 06:30:09 on dist-test-slave-mjjr"
server_key: "2fe6c06b2de5c00eac3943b6aa54bce9"
server_key_iv: "ed9bb0ec2a557c4d2e4dd64945e1b5b7"
server_key_version: "encryptionkey@0"
I20260809 06:30:09.569449 2327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:09.585239 2327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:09.589138 2327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:09.589388 2327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:09.590286 2327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:09.591573 2327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:09.591691 2327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:09.591821 2327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:09.591873 2327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:09.601246 2129 heartbeater.cc:503] Master 127.8.131.62:36939 was elected leader, sending a full tablet report...
I20260809 06:30:09.626788 2327 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:41415
I20260809 06:30:09.626868 2508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:41415 every 8 connection(s)
I20260809 06:30:09.628121 2327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:30:09.631826 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 2327
I20260809 06:30:09.632025 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:30:09.632570 8716 external_mini_cluster.cc:1428] Setting key 05ccea4107cfea248613699c807e96c3
Aug 09 06:30:09 dist-test-slave-mjjr krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257009, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:09.648664 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.630596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:50619 (local address 127.8.131.62:36939)
0809 06:30:09.631059 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:09.631064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:09.632261 (+ 1197us) server_negotiation.cc:407] Connection header received
0809 06:30:09.634339 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:09.634343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:09.634420 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:09.634546 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:09.636196 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.636972 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.638909 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.639343 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.639496 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.642855 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:09.642865 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:09.642869 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:09.642899 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:09.645343 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:09.646151 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:09.646157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:09.646160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:09.646331 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:09.646852 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:09.646856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:09.646858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:09.647183 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:09.647329 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:09.648214 (+ 885us) server_negotiation.cc:299] Negotiation successful
0809 06:30:09.648378 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":206,"threads_started":1}
I20260809 06:30:09.649581 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.631181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:36939 (local address 127.8.131.3:50619)
0809 06:30:09.632071 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:09.632134 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:09.634079 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:09.634807 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:09.634817 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:09.635347 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:09.636031 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.636048 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.637144 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.637148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.638714 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.638730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.639583 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.639591 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.639859 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.640477 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:30:09.640498 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:09.642656 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:09.645480 (+ 2824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:09.645489 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:09.645502 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:09.645988 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:09.646444 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:09.646448 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:09.646451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:09.646716 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:09.647357 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:09.647364 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:09.647777 (+ 413us) client_negotiation.cc:769] Sending connection context
0809 06:30:09.648106 (+ 329us) client_negotiation.cc:240] Negotiation successful
0809 06:30:09.648471 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":414,"threads_started":1}
I20260809 06:30:09.650867 2510 heartbeater.cc:347] Connected to a master server at 127.8.131.62:36939
I20260809 06:30:09.651175 2510 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:09.651940 2510 heartbeater.cc:511] Master 127.8.131.62:36939 requested a full tablet report, sending...
I20260809 06:30:09.653348 1862 ts_manager.cc:194] Registered new tserver with Master: bd73138796e6438c905eb4f1888895ba (127.8.131.3:41415)
I20260809 06:30:09.654241 1862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:50619
I20260809 06:30:09.660221 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:30:09.677263 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.664800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.8.131.62:36939)
0809 06:30:09.664960 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:09.664964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:09.665463 (+ 499us) server_negotiation.cc:407] Connection header received
0809 06:30:09.665579 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:09.665583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:09.665680 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:09.665766 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:09.667139 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.667922 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.669178 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.669555 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.669642 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.672041 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:09.672045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:09.672048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:09.672076 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:09.674554 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:09.675255 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:09.675259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:09.675261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:09.675369 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:09.675770 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:09.675775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:09.675779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:09.676104 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:09.676382 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:09.676731 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 06:30:09.676951 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 06:30:09.681159 1862 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:30:09.688568 1862 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:30:09.711733 2529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.700138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:41415 (local address 127.0.0.1:49438)
0809 06:30:09.702047 (+ 1909us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:09.702088 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:09.702235 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:09.704044 (+ 1809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:09.704051 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:09.704082 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:09.704481 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.704494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.706687 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.706693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.708935 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.708947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.710699 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.710707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.710911 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.710964 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 06:30:09.711089 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 06:30:09.711296 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1676,"thread_start_us":3458,"threads_started":1}
I20260809 06:30:09.712090 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.701190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43806 (local address 127.8.131.1:45253)
0809 06:30:09.703414 (+ 2224us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:09.703425 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:09.704047 (+ 622us) server_negotiation.cc:407] Connection header received
0809 06:30:09.704380 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:09.704387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:09.704625 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:09.704820 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:09.705594 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.707199 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.709842 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.710935 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.711093 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.711313 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:09.711542 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 06:30:09.711738 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2011,"thread_start_us":1822,"threads_started":1}
I20260809 06:30:09.712651 2526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.700504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:45253 (local address 127.0.0.1:43806)
0809 06:30:09.703928 (+ 3424us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:09.703947 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:09.704166 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:09.704805 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:09.704810 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:09.704834 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:09.705371 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.705387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.707335 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.707341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.709689 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.709702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.711166 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.711173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.711353 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.711384 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 06:30:09.711480 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 06:30:09.712228 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3293,"mutex_wait_us":1,"thread_start_us":1187,"threads_started":1}
I20260809 06:30:09.712991 2530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.700421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49438 (local address 127.8.131.3:41415)
0809 06:30:09.703393 (+ 2972us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:09.703402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:09.703435 (+ 33us) server_negotiation.cc:407] Connection header received
0809 06:30:09.703537 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:09.703544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:09.703816 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:09.704681 (+ 865us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:09.704967 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.706512 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.709107 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.710464 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.710608 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.710854 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:09.711720 (+ 866us) server_negotiation.cc:299] Negotiation successful
0809 06:30:09.712554 (+ 834us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2413,"thread_start_us":2183,"threads_started":1}
I20260809 06:30:09.715116 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.704812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:45519 (local address 127.0.0.1:48648)
0809 06:30:09.706201 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:09.706215 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:09.706358 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:09.706985 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:09.706989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:09.707014 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:09.707607 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.707627 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.709576 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.709581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.712339 (+ 2758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.712354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.714333 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.714340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.714504 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.714538 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:30:09.714646 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 06:30:09.714779 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1282,"mutex_wait_us":512,"thread_start_us":1127,"threads_started":1}
I20260809 06:30:09.715173 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.704240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48648 (local address 127.8.131.2:45519)
0809 06:30:09.705732 (+ 1492us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:09.705744 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:09.706322 (+ 578us) server_negotiation.cc:407] Connection header received
0809 06:30:09.706550 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:09.706555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:09.706792 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:09.706984 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:09.707849 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.709414 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.712661 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.714111 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.714249 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.714439 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:09.714706 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 06:30:09.714880 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":294,"threads_started":1}
I20260809 06:30:09.716655 2037 tablet_service.cc:1467] Processing CreateTablet for tablet eab302c066a6488ba57df190ee2bb049 (DEFAULT_TABLE table=test-table [id=2f393cf3f8764670b459633428e26142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:09.716487 2412 tablet_service.cc:1467] Processing CreateTablet for tablet eab302c066a6488ba57df190ee2bb049 (DEFAULT_TABLE table=test-table [id=2f393cf3f8764670b459633428e26142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:09.718215 2037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eab302c066a6488ba57df190ee2bb049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:09.718235 2412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eab302c066a6488ba57df190ee2bb049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:09.718381 2223 tablet_service.cc:1467] Processing CreateTablet for tablet eab302c066a6488ba57df190ee2bb049 (DEFAULT_TABLE table=test-table [id=2f393cf3f8764670b459633428e26142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:09.719697 2223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eab302c066a6488ba57df190ee2bb049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:09.729429 2537 tablet_bootstrap.cc:492] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: Bootstrap starting.
I20260809 06:30:09.733107 2540 tablet_bootstrap.cc:492] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: Bootstrap starting.
I20260809 06:30:09.733595 2537 tablet_bootstrap.cc:654] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:09.734781 2542 tablet_bootstrap.cc:492] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: Bootstrap starting.
I20260809 06:30:09.735139 2537 log.cc:824] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:09.736593 2540 tablet_bootstrap.cc:654] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:09.737533 2540 log.cc:824] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:09.738376 2542 tablet_bootstrap.cc:654] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:09.738613 2537 tablet_bootstrap.cc:492] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: No bootstrap required, opened a new log
I20260809 06:30:09.738897 2537 ts_tablet_manager.cc:1397] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 06:30:09.739573 2542 log.cc:824] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:09.741341 2540 tablet_bootstrap.cc:492] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: No bootstrap required, opened a new log
I20260809 06:30:09.741626 2540 ts_tablet_manager.cc:1397] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 06:30:09.744033 2542 tablet_bootstrap.cc:492] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: No bootstrap required, opened a new log
I20260809 06:30:09.744287 2542 ts_tablet_manager.cc:1397] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260809 06:30:09.748793 2540 raft_consensus.cc:348] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } }
I20260809 06:30:09.749164 2540 raft_consensus.cc:374] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:09.749591 2540 raft_consensus.cc:729] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5be3f2f542540b08b3f309f34d35f59, State: Initialized, Role: FOLLOWER
I20260809 06:30:09.749773 2537 raft_consensus.cc:348] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } }
I20260809 06:30:09.750175 2537 raft_consensus.cc:374] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:09.750303 2537 raft_consensus.cc:729] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd73138796e6438c905eb4f1888895ba, State: Initialized, Role: FOLLOWER
I20260809 06:30:09.750761 2540 consensus_queue.cc:260] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } }
I20260809 06:30:09.750855 2537 consensus_queue.cc:260] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } }
I20260809 06:30:09.752674 2540 ts_tablet_manager.cc:1428] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260809 06:30:09.753590 2510 heartbeater.cc:503] Master 127.8.131.62:36939 was elected leader, sending a full tablet report...
I20260809 06:30:09.754262 2537 ts_tablet_manager.cc:1428] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: Time spent starting tablet: real 0.015s user 0.013s sys 0.004s
I20260809 06:30:09.755506 2542 raft_consensus.cc:348] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } }
I20260809 06:30:09.756008 2542 raft_consensus.cc:374] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:09.756125 2542 raft_consensus.cc:729] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c562d6c8b894fdbb76de657501871f0, State: Initialized, Role: FOLLOWER
I20260809 06:30:09.756747 2542 consensus_queue.cc:260] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } }
I20260809 06:30:09.757895 2321 heartbeater.cc:503] Master 127.8.131.62:36939 was elected leader, sending a full tablet report...
I20260809 06:30:09.758616 2542 ts_tablet_manager.cc:1428] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
W20260809 06:30:09.823298 2132 tablet.cc:2369] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:30:09.851747 2322 tablet.cc:2369] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:30:09.879786 2512 tablet.cc:2369] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:30:09.936297 2547 raft_consensus.cc:482] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:30:09.936625 2547 raft_consensus.cc:504] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } }
I20260809 06:30:09.938482 2547 leader_election.cc:288] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd73138796e6438c905eb4f1888895ba (127.8.131.3:41415), 8c562d6c8b894fdbb76de657501871f0 (127.8.131.2:45519)
I20260809 06:30:09.945885 2530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.938872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:54105 (local address 127.8.131.3:41415)
0809 06:30:09.939103 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:09.939111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:09.939567 (+ 456us) server_negotiation.cc:407] Connection header received
0809 06:30:09.939853 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:09.939861 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:09.939951 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:09.940052 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:09.940949 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.941907 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.943966 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.944874 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.944974 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.945050 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:09.945493 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 06:30:09.945626 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 06:30:09.945950 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.938681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:41415 (local address 127.8.131.1:54105)
0809 06:30:09.939422 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:09.939438 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:09.939616 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:09.940245 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:09.940250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:09.940301 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:09.940780 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.940796 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.942047 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.942052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.943812 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.943824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.945096 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.945103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.945303 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.945340 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 06:30:09.945434 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 06:30:09.945529 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":3,"thread_start_us":379,"threads_started":1}
I20260809 06:30:09.946149 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.940060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:35257 (local address 127.8.131.2:45519)
0809 06:30:09.940301 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:09.940307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:09.940438 (+ 131us) server_negotiation.cc:407] Connection header received
0809 06:30:09.940593 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:09.940598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:09.940682 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:09.940838 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:09.941488 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.942344 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.944277 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.945134 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.945272 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.945360 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:09.945730 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 06:30:09.945875 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 06:30:09.946449 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:09.939566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:45519 (local address 127.8.131.1:35257)
0809 06:30:09.940336 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:09.940350 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:09.940467 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:09.940878 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:09.940885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:09.940915 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:09.941330 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.941341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.942505 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.942509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.944112 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:09.944128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:09.945357 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:09.945361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:09.945550 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:09.945584 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 06:30:09.945686 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 06:30:09.945814 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":87,"thread_start_us":451,"threads_started":1}
I20260809 06:30:09.946940 2437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab302c066a6488ba57df190ee2bb049" candidate_uuid: "a5be3f2f542540b08b3f309f34d35f59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd73138796e6438c905eb4f1888895ba" is_pre_election: true
I20260809 06:30:09.947465 2437 raft_consensus.cc:2454] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5be3f2f542540b08b3f309f34d35f59 in term 0.
I20260809 06:30:09.947434 2256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab302c066a6488ba57df190ee2bb049" candidate_uuid: "a5be3f2f542540b08b3f309f34d35f59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c562d6c8b894fdbb76de657501871f0" is_pre_election: true
I20260809 06:30:09.947991 2256 raft_consensus.cc:2454] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5be3f2f542540b08b3f309f34d35f59 in term 0.
I20260809 06:30:09.948141 1973 leader_election.cc:302] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5be3f2f542540b08b3f309f34d35f59, bd73138796e6438c905eb4f1888895ba; no voters:
I20260809 06:30:09.948607 2547 raft_consensus.cc:2781] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:30:09.948782 2547 raft_consensus.cc:482] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:30:09.948915 2547 raft_consensus.cc:3037] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:09.951399 2547 raft_consensus.cc:504] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } }
I20260809 06:30:09.952013 2547 leader_election.cc:288] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [CANDIDATE]: Term 1 election: Requested vote from peers bd73138796e6438c905eb4f1888895ba (127.8.131.3:41415), 8c562d6c8b894fdbb76de657501871f0 (127.8.131.2:45519)
I20260809 06:30:09.952486 2437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab302c066a6488ba57df190ee2bb049" candidate_uuid: "a5be3f2f542540b08b3f309f34d35f59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd73138796e6438c905eb4f1888895ba"
I20260809 06:30:09.952543 2256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eab302c066a6488ba57df190ee2bb049" candidate_uuid: "a5be3f2f542540b08b3f309f34d35f59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c562d6c8b894fdbb76de657501871f0"
I20260809 06:30:09.952668 2437 raft_consensus.cc:3037] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:09.952747 2256 raft_consensus.cc:3037] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:09.954978 2256 raft_consensus.cc:2454] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5be3f2f542540b08b3f309f34d35f59 in term 1.
I20260809 06:30:09.955247 2437 raft_consensus.cc:2454] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5be3f2f542540b08b3f309f34d35f59 in term 1.
I20260809 06:30:09.955384 1973 leader_election.cc:302] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c562d6c8b894fdbb76de657501871f0, a5be3f2f542540b08b3f309f34d35f59; no voters:
I20260809 06:30:09.955698 2547 raft_consensus.cc:2781] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:09.956135 2547 raft_consensus.cc:686] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 1 LEADER]: Becoming Leader. State: Replica: a5be3f2f542540b08b3f309f34d35f59, State: Running, Role: LEADER
I20260809 06:30:09.956620 2547 consensus_queue.cc:237] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } }
I20260809 06:30:09.961381 1862 catalog_manager.cc:5495] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5be3f2f542540b08b3f309f34d35f59 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "a5be3f2f542540b08b3f309f34d35f59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5be3f2f542540b08b3f309f34d35f59" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 45253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260809 06:30:10.006126 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:10.000964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.8.131.1:45253)
0809 06:30:10.001164 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:10.001168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:10.001184 (+ 16us) server_negotiation.cc:407] Connection header received
0809 06:30:10.001282 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:10.001285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:10.001369 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:10.001514 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:30:10.001834 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:10.002542 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:10.004039 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:10.004578 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:10.004686 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:10.005126 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:30:10.005624 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:30:10.005732 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:10.005864 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 06:30:10.005928 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 06:30:10.014231 2437 raft_consensus.cc:1261] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [term 1 FOLLOWER]: Refusing update from remote peer a5be3f2f542540b08b3f309f34d35f59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:10.014228 2256 raft_consensus.cc:1261] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [term 1 FOLLOWER]: Refusing update from remote peer a5be3f2f542540b08b3f309f34d35f59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:10.015208 2557 consensus_queue.cc:1035] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c562d6c8b894fdbb76de657501871f0" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:10.015599 2547 consensus_queue.cc:1035] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd73138796e6438c905eb4f1888895ba" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 41415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:10.027354 2560 mvcc.cc:205] Tried to move back new op lower bound from 7316508713008656384 to 7316508712786157568. Current Snapshot: MvccSnapshot[applied={T|T < 7316508713008656384}]
I20260809 06:30:10.030151 2561 mvcc.cc:205] Tried to move back new op lower bound from 7316508713008656384 to 7316508712786157568. Current Snapshot: MvccSnapshot[applied={T|T < 7316508713008656384}]
I20260809 06:30:10.041352 1858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:10.041666 1858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:10.045905 1858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8abbbd5fd9824a7b95b9ac6a1251fc98: Sending DeleteTablet for 3 replicas of tablet eab302c066a6488ba57df190ee2bb049
I20260809 06:30:10.047070 2037 tablet_service.cc:1514] Processing DeleteTablet for tablet eab302c066a6488ba57df190ee2bb049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:10 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:43806
I20260809 06:30:10.047348 2223 tablet_service.cc:1514] Processing DeleteTablet for tablet eab302c066a6488ba57df190ee2bb049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:10 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:48648
I20260809 06:30:10.047581 2412 tablet_service.cc:1514] Processing DeleteTablet for tablet eab302c066a6488ba57df190ee2bb049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:10 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:49438
I20260809 06:30:10.052356 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 1953
I20260809 06:30:10.065649 2581 tablet_replica.cc:330] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: stopping tablet replica
I20260809 06:30:10.066238 2581 raft_consensus.cc:2229] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:10.067058 2581 raft_consensus.cc:2258] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:10.069751 2581 ts_tablet_manager.cc:1905] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 06:30:10 dist-test-slave-mjjr krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:30:10.073835 2580 tablet_replica.cc:330] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: stopping tablet replica
I20260809 06:30:10.074371 2580 raft_consensus.cc:2229] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:10.074980 2580 raft_consensus.cc:2258] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:10.075531 2579 tablet_replica.cc:330] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: stopping tablet replica
I20260809 06:30:10.075989 2579 raft_consensus.cc:2229] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:30:10.076637 2579 raft_consensus.cc:2258] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:10.077546 2580 ts_tablet_manager.cc:1905] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:10.079677 2579 ts_tablet_manager.cc:1905] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:10.080883 2581 ts_tablet_manager.cc:1918] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:10.081040 2581 log.cc:1196] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/eab302c066a6488ba57df190ee2bb049
I20260809 06:30:10.081581 2581 ts_tablet_manager.cc:1939] T eab302c066a6488ba57df190ee2bb049 P bd73138796e6438c905eb4f1888895ba: Deleting consensus metadata
I20260809 06:30:10.083439 1840 catalog_manager.cc:4841] TS bd73138796e6438c905eb4f1888895ba (127.8.131.3:41415): tablet eab302c066a6488ba57df190ee2bb049 (table test-table [id=2f393cf3f8764670b459633428e26142]) successfully deleted
I20260809 06:30:10.084929 2580 ts_tablet_manager.cc:1918] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:10.085160 2580 log.cc:1196] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/eab302c066a6488ba57df190ee2bb049
I20260809 06:30:10.085769 2580 ts_tablet_manager.cc:1939] T eab302c066a6488ba57df190ee2bb049 P 8c562d6c8b894fdbb76de657501871f0: Deleting consensus metadata
I20260809 06:30:10.087409 2579 ts_tablet_manager.cc:1918] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:10.087589 2579 log.cc:1196] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/eab302c066a6488ba57df190ee2bb049
I20260809 06:30:10.087567 1840 catalog_manager.cc:4841] TS 8c562d6c8b894fdbb76de657501871f0 (127.8.131.2:45519): tablet eab302c066a6488ba57df190ee2bb049 (table test-table [id=2f393cf3f8764670b459633428e26142]) successfully deleted
I20260809 06:30:10.088166 2579 ts_tablet_manager.cc:1939] T eab302c066a6488ba57df190ee2bb049 P a5be3f2f542540b08b3f309f34d35f59: Deleting consensus metadata
I20260809 06:30:10.089928 1843 catalog_manager.cc:4841] TS a5be3f2f542540b08b3f309f34d35f59 (127.8.131.1:45253): tablet eab302c066a6488ba57df190ee2bb049 (table test-table [id=2f393cf3f8764670b459633428e26142]) successfully deleted
I20260809 06:30:10.092057 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:10.054825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.8.131.1:45253)
0809 06:30:10.055033 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:10.055039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:10.056126 (+ 1087us) server_negotiation.cc:407] Connection header received
0809 06:30:10.056885 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:10.056890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:10.057018 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:10.057113 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:10.061778 (+ 4665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:10.063572 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:10.068164 (+ 4592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:10.068867 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:10.068968 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:10.084070 (+ 15102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:10.084083 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:10.084100 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:10.084157 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:10.088800 (+ 4643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:10.089840 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:10.089848 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:10.089851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:10.090070 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:10.090500 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:10.090503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:10.090505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:10.090881 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:10.091097 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:10.091498 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 06:30:10.091746 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:30:10.092764 2121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:10.193778 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 1953
I20260809 06:30:10.217546 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 2139
Aug 09 06:30:10 dist-test-slave-mjjr krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:30:10.232244 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:10.218503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48656 (local address 127.8.131.2:45519)
0809 06:30:10.218674 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:10.218679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:10.218712 (+ 33us) server_negotiation.cc:407] Connection header received
0809 06:30:10.218768 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:10.218771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:10.218861 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:10.218940 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:10.220263 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:10.221007 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:10.222231 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:10.222606 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:10.222694 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:10.227003 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:10.227018 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:10.227034 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:10.227077 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:10.229672 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:10.230403 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:10.230410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:10.230412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:10.230535 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:10.230842 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:10.230845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:10.230846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:10.231206 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:10.231418 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:10.231819 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 06:30:10.231987 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:30:10.232985 2315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:10.321800 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 2139
I20260809 06:30:10.345018 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 2327
Aug 09 06:30:10 dist-test-slave-mjjr krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:30:10.363554 2530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:10.345795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49444 (local address 127.8.131.3:41415)
0809 06:30:10.345963 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:10.345967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:10.345985 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:30:10.346111 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:10.346115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:10.346190 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:10.346275 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:10.347711 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:10.348421 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:10.349729 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:10.350094 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:10.350185 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:10.358150 (+ 7965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:10.358164 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:10.358180 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:10.358213 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:10.361135 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:10.361747 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:10.361754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:10.361756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:10.361908 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:10.362222 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:10.362225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:10.362227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:10.362569 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:10.362777 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:10.363015 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 06:30:10.363172 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 06:30:10.364312 2502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:10.453544 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 2327
I20260809 06:30:10.473628 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 1814
I20260809 06:30:10.474771 1904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:10.585340 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 1814
2026-08-09T06:30:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4736 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T06:30:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:10Z Disabled control of system clock
I20260809 06:30:10.677369 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38855
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:38855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 06:30:11.015810 2595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:11.016157 2595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:11.016309 2595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:11.023504 2595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:11.023649 2595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:11.023741 2595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:11.023837 2595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:11.023926 2595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:11.033111 2595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:38855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:11.034422 2595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:11.036329 2595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:11.048380 2606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:11.048345 2603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:11.048908 2602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:11.049609 2595 server_base.cc:1034] running on GCE node
I20260809 06:30:11.050357 2595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:11.051785 2595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:11.053020 2595 hybrid_clock.cc:648] HybridClock initialized: now 1786257011052984 us; error 49 us; skew 500 ppm
I20260809 06:30:11.053375 2595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:11.055581 2595 webserver.cc:460] Webserver started at http://127.8.131.62:40641/ using document root <none> and password file <none>
I20260809 06:30:11.056408 2595 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:11.056555 2595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:11.056895 2595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:11.059928 2595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "c29e95c659b2475a8f27189c6c740f74"
format_stamp: "Formatted at 2026-08-09 06:30:11 on dist-test-slave-mjjr"
server_key: "738de5dab1779917b7a1c0fd3da31211"
server_key_iv: "9b0a40502690507a0d5a796574d08adf"
server_key_version: "encryptionkey@0"
I20260809 06:30:11.060757 2595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "c29e95c659b2475a8f27189c6c740f74"
format_stamp: "Formatted at 2026-08-09 06:30:11 on dist-test-slave-mjjr"
server_key: "738de5dab1779917b7a1c0fd3da31211"
server_key_iv: "9b0a40502690507a0d5a796574d08adf"
server_key_version: "encryptionkey@0"
I20260809 06:30:11.065909 2595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:30:11.069423 2616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:11.070708 2595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:30:11.070910 2595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "c29e95c659b2475a8f27189c6c740f74"
format_stamp: "Formatted at 2026-08-09 06:30:11 on dist-test-slave-mjjr"
server_key: "738de5dab1779917b7a1c0fd3da31211"
server_key_iv: "9b0a40502690507a0d5a796574d08adf"
server_key_version: "encryptionkey@0"
I20260809 06:30:11.071134 2595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:11.087832 2595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:11.088614 2595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:11.088904 2595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:11.107504 2595 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:38855
I20260809 06:30:11.107537 2698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:38855 every 8 connection(s)
I20260809 06:30:11.108820 2595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:11.112442 2699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:11.112948 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 2595
I20260809 06:30:11.113160 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:11.113783 8716 external_mini_cluster.cc:1428] Setting key 59a7cff09b5db33d9d8bead71789383b
I20260809 06:30:11.126374 2699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74: Bootstrap starting.
I20260809 06:30:11.128198 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:11.115745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58448 (local address 127.8.131.62:38855)
0809 06:30:11.116778 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:11.116788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:11.116833 (+ 45us) server_negotiation.cc:407] Connection header received
0809 06:30:11.118750 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:11.118771 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:11.119209 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:11.119547 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:11.120194 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:11.121487 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:11.123464 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:11.124209 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:11.124350 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:11.125525 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:11.125531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:11.125542 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:11.126090 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:11.126189 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:11.126224 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:11.126363 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:11.126639 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 06:30:11.126964 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":398,"threads_started":1}
I20260809 06:30:11.129993 2699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:11.131126 2699 log.cc:824] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:11.133677 2699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74: No bootstrap required, opened a new log
I20260809 06:30:11.140854 2699 raft_consensus.cc:348] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c29e95c659b2475a8f27189c6c740f74" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38855 } }
I20260809 06:30:11.141237 2699 raft_consensus.cc:374] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:11.141352 2699 raft_consensus.cc:729] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29e95c659b2475a8f27189c6c740f74, State: Initialized, Role: FOLLOWER
I20260809 06:30:11.141887 2699 consensus_queue.cc:260] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c29e95c659b2475a8f27189c6c740f74" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38855 } }
I20260809 06:30:11.142141 2699 raft_consensus.cc:388] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:11.142290 2699 raft_consensus.cc:482] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:11.142474 2699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:11.144726 2699 raft_consensus.cc:504] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c29e95c659b2475a8f27189c6c740f74" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38855 } }
I20260809 06:30:11.145226 2699 leader_election.cc:302] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c29e95c659b2475a8f27189c6c740f74; no voters:
I20260809 06:30:11.145881 2699 leader_election.cc:288] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:11.146021 2706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:11.146916 2706 raft_consensus.cc:686] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [term 1 LEADER]: Becoming Leader. State: Replica: c29e95c659b2475a8f27189c6c740f74, State: Running, Role: LEADER
I20260809 06:30:11.147455 2706 consensus_queue.cc:237] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c29e95c659b2475a8f27189c6c740f74" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38855 } }
I20260809 06:30:11.148267 2699 sys_catalog.cc:564] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:11.150374 2710 sys_catalog.cc:455] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c29e95c659b2475a8f27189c6c740f74. Latest consensus state: current_term: 1 leader_uuid: "c29e95c659b2475a8f27189c6c740f74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c29e95c659b2475a8f27189c6c740f74" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38855 } } }
I20260809 06:30:11.150496 2707 sys_catalog.cc:455] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c29e95c659b2475a8f27189c6c740f74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c29e95c659b2475a8f27189c6c740f74" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38855 } } }
I20260809 06:30:11.150897 2707 sys_catalog.cc:458] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:11.150810 2710 sys_catalog.cc:458] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:11.153134 2715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:11.158385 2715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:11.167389 2715 catalog_manager.cc:1347] Generated new cluster ID: 050b23bd7aaf47fd9a6e293092a22dda
I20260809 06:30:11.167537 2715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:11.188764 2715 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:11.190670 2715 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:11.201817 2715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74: Generated new TSK 0
I20260809 06:30:11.202773 2715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:11.215616 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38855
--builtin_ntp_servers=127.8.131.20:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 06:30:11.560222 2733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:11.560628 2733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:11.560798 2733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:11.568331 2733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:11.568429 2733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:11.568540 2733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:30:11.577968 2733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:11.579161 2733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:11.581101 2733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:11.594499 2739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:11.594451 2740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:11.594537 2743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:11.595345 2733 server_base.cc:1034] running on GCE node
I20260809 06:30:11.596036 2733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:11.596930 2733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:11.598136 2733 hybrid_clock.cc:648] HybridClock initialized: now 1786257011598095 us; error 33 us; skew 500 ppm
I20260809 06:30:11.598481 2733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:11.600729 2733 webserver.cc:460] Webserver started at http://127.8.131.1:41337/ using document root <none> and password file <none>
I20260809 06:30:11.601431 2733 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:11.601553 2733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:11.601890 2733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:11.604772 2733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "5933e5225d15460e829f9fe68c4453c2"
format_stamp: "Formatted at 2026-08-09 06:30:11 on dist-test-slave-mjjr"
server_key: "82c907ae3a6792f740f59d542d62f233"
server_key_iv: "a053902da6cb65a6b96eb1336c72a0e5"
server_key_version: "encryptionkey@0"
I20260809 06:30:11.605599 2733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "5933e5225d15460e829f9fe68c4453c2"
format_stamp: "Formatted at 2026-08-09 06:30:11 on dist-test-slave-mjjr"
server_key: "82c907ae3a6792f740f59d542d62f233"
server_key_iv: "a053902da6cb65a6b96eb1336c72a0e5"
server_key_version: "encryptionkey@0"
I20260809 06:30:11.610706 2733 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:30:11.614115 2749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:11.615406 2733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:30:11.615581 2733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "5933e5225d15460e829f9fe68c4453c2"
format_stamp: "Formatted at 2026-08-09 06:30:11 on dist-test-slave-mjjr"
server_key: "82c907ae3a6792f740f59d542d62f233"
server_key_iv: "a053902da6cb65a6b96eb1336c72a0e5"
server_key_version: "encryptionkey@0"
I20260809 06:30:11.615835 2733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:11.636500 2733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:11.637239 2733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:11.637472 2733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:11.638381 2733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:11.639884 2733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:11.639973 2733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:11.640053 2733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:11.640107 2733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:11.673746 2733 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:46565
I20260809 06:30:11.673851 2915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:46565 every 8 connection(s)
I20260809 06:30:11.675112 2733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:30:11.676774 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 2733
I20260809 06:30:11.676990 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:30:11.677579 8716 external_mini_cluster.cc:1428] Setting key a8e32d84104db8dd6adfb77e0748d819
I20260809 06:30:11.681003 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38855
--builtin_ntp_servers=127.8.131.20:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 06:30:11.690014 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:11.677957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:50131 (local address 127.8.131.62:38855)
0809 06:30:11.678512 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:11.678518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:11.679819 (+ 1301us) server_negotiation.cc:407] Connection header received
0809 06:30:11.682672 (+ 2853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:11.682680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:11.682781 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:11.682891 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:11.684027 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:11.685187 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:11.686788 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:11.687522 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:11.687692 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:11.689038 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:11.689046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:11.689054 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:11.689080 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:11.689135 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:11.689154 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:11.689249 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:11.689471 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 06:30:11.689581 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":278,"threads_started":1}
I20260809 06:30:11.690739 2920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:11.678601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38855 (local address 127.8.131.1:50131)
0809 06:30:11.679611 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:11.679712 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:11.682244 (+ 2532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:11.683106 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:11.683114 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:11.683196 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:11.683851 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:11.683872 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:11.685359 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:11.685364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:11.686637 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:11.686650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:11.687791 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:11.687797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:11.687986 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:11.688714 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:30:11.688717 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:11.688754 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:30:11.688838 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:11.689252 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:11.689272 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:11.689277 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 06:30:11.689411 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 06:30:11.689692 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":313,"threads_started":1}
I20260809 06:30:11.692412 2916 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38855
I20260809 06:30:11.692730 2916 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:11.693496 2916 heartbeater.cc:511] Master 127.8.131.62:38855 requested a full tablet report, sending...
I20260809 06:30:11.695729 2644 ts_manager.cc:194] Registered new tserver with Master: 5933e5225d15460e829f9fe68c4453c2 (127.8.131.1:46565)
I20260809 06:30:11.697962 2644 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.1:50131
W20260809 06:30:12.031776 2921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:12.032155 2921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:12.032270 2921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:12.039746 2921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:12.039858 2921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:12.039981 2921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:30:12.049515 2921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:12.050788 2921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:12.052781 2921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:12.066208 2928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:12.066264 2933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:12.066296 2929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:12.069418 2921 server_base.cc:1034] running on GCE node
I20260809 06:30:12.070137 2921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:12.071172 2921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:12.072409 2921 hybrid_clock.cc:648] HybridClock initialized: now 1786257012072344 us; error 66 us; skew 500 ppm
I20260809 06:30:12.072826 2921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:12.075330 2921 webserver.cc:460] Webserver started at http://127.8.131.2:38977/ using document root <none> and password file <none>
I20260809 06:30:12.076164 2921 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:12.076267 2921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:12.076560 2921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:12.079519 2921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "44a2546b1d70437e93bf65a97043ba65"
format_stamp: "Formatted at 2026-08-09 06:30:12 on dist-test-slave-mjjr"
server_key: "156ddb353909f34847b772867afe92ce"
server_key_iv: "b3e658f605a285fb48cbd02d2db9a07e"
server_key_version: "encryptionkey@0"
I20260809 06:30:12.080431 2921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "44a2546b1d70437e93bf65a97043ba65"
format_stamp: "Formatted at 2026-08-09 06:30:12 on dist-test-slave-mjjr"
server_key: "156ddb353909f34847b772867afe92ce"
server_key_iv: "b3e658f605a285fb48cbd02d2db9a07e"
server_key_version: "encryptionkey@0"
I20260809 06:30:12.085906 2921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:30:12.089495 2945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:12.090844 2921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:30:12.091130 2921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "44a2546b1d70437e93bf65a97043ba65"
format_stamp: "Formatted at 2026-08-09 06:30:12 on dist-test-slave-mjjr"
server_key: "156ddb353909f34847b772867afe92ce"
server_key_iv: "b3e658f605a285fb48cbd02d2db9a07e"
server_key_version: "encryptionkey@0"
I20260809 06:30:12.091367 2921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:12.116235 2921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:12.117364 2921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:12.117678 2921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:12.118856 2921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:12.120867 2921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:12.120985 2921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:12.121099 2921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:12.121178 2921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:12.157195 2921 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:39343
I20260809 06:30:12.157260 3106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:39343 every 8 connection(s)
I20260809 06:30:12.158627 2921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:30:12.164768 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 2921
I20260809 06:30:12.164975 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:30:12.165560 8716 external_mini_cluster.cc:1428] Setting key 3f47f11f1323d9626d9d58ac50d4b8e4
I20260809 06:30:12.168711 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38855
--builtin_ntp_servers=127.8.131.20:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 06:30:12.172695 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.161323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:41199 (local address 127.8.131.62:38855)
0809 06:30:12.161500 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:12.161505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:12.162904 (+ 1399us) server_negotiation.cc:407] Connection header received
0809 06:30:12.165068 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:12.165077 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:12.165162 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:12.165254 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:12.166593 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.167346 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.169759 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.170169 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.170272 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.171920 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:12.171924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:12.171928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:12.171947 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:12.172005 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:12.172017 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:12.172089 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:12.172352 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 06:30:12.172430 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:30:12.174189 3110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.161879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38855 (local address 127.8.131.2:41199)
0809 06:30:12.162749 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:12.162790 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:12.164632 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:12.165524 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:12.165537 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:12.165658 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:12.166396 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.166415 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.167505 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.167518 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.169609 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.169627 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.170406 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.170416 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.170753 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.171527 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:30:12.171532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:12.171588 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:30:12.171721 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:12.172122 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:12.172138 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:12.172149 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 06:30:12.172324 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 06:30:12.172771 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":285,"threads_started":1}
I20260809 06:30:12.175623 3107 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38855
I20260809 06:30:12.176029 3107 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:12.176766 3107 heartbeater.cc:511] Master 127.8.131.62:38855 requested a full tablet report, sending...
I20260809 06:30:12.178231 2644 ts_manager.cc:194] Registered new tserver with Master: 44a2546b1d70437e93bf65a97043ba65 (127.8.131.2:39343)
I20260809 06:30:12.179050 2644 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.2:41199
W20260809 06:30:12.510968 3111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:12.511256 3111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:12.511400 3111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:12.518527 3111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:12.518643 3111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:12.518831 3111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:30:12.527992 3111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:12.529320 3111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:12.531075 3111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:12.544281 3117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:12.544248 3118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:12.544333 3122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:12.544981 3111 server_base.cc:1034] running on GCE node
I20260809 06:30:12.545555 3111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:12.546487 3111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:12.547747 3111 hybrid_clock.cc:648] HybridClock initialized: now 1786257012547693 us; error 46 us; skew 500 ppm
I20260809 06:30:12.548074 3111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:12.550566 3111 webserver.cc:460] Webserver started at http://127.8.131.3:33243/ using document root <none> and password file <none>
I20260809 06:30:12.551260 3111 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:12.551396 3111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:12.551805 3111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:12.554729 3111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "b7ffadc0f66f4f5db30d808a8363f0df"
format_stamp: "Formatted at 2026-08-09 06:30:12 on dist-test-slave-mjjr"
server_key: "597437646ca1e6263dcbea851f05d780"
server_key_iv: "c359d24bd881639f510980293cd6e29a"
server_key_version: "encryptionkey@0"
I20260809 06:30:12.555562 3111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "b7ffadc0f66f4f5db30d808a8363f0df"
format_stamp: "Formatted at 2026-08-09 06:30:12 on dist-test-slave-mjjr"
server_key: "597437646ca1e6263dcbea851f05d780"
server_key_iv: "c359d24bd881639f510980293cd6e29a"
server_key_version: "encryptionkey@0"
I20260809 06:30:12.560576 3111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:30:12.564042 3131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:12.565272 3111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:30:12.565442 3111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "b7ffadc0f66f4f5db30d808a8363f0df"
format_stamp: "Formatted at 2026-08-09 06:30:12 on dist-test-slave-mjjr"
server_key: "597437646ca1e6263dcbea851f05d780"
server_key_iv: "c359d24bd881639f510980293cd6e29a"
server_key_version: "encryptionkey@0"
I20260809 06:30:12.565677 3111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:12.596910 3111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:12.597769 3111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:12.598062 3111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:12.599107 3111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:12.600653 3111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:12.600785 3111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:12.600922 3111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:12.601013 3111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:12.638041 3111 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:45403
I20260809 06:30:12.638136 3296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:45403 every 8 connection(s)
I20260809 06:30:12.639370 3111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:30:12.641106 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 3111
I20260809 06:30:12.641309 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:30:12.641947 8716 external_mini_cluster.cc:1428] Setting key 735e1d4e468bcc0c17e1c0af352ffdaa
I20260809 06:30:12.653116 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.642617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:33561 (local address 127.8.131.62:38855)
0809 06:30:12.642809 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:12.642813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:12.644253 (+ 1440us) server_negotiation.cc:407] Connection header received
0809 06:30:12.646443 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:12.646447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:12.646531 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:12.646615 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:12.647739 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.648609 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.650190 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.650628 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.650726 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.652336 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:12.652340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:12.652343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:12.652362 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:12.652402 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:12.652414 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:12.652493 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:12.652702 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 06:30:12.652783 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:30:12.654315 3299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.642977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38855 (local address 127.8.131.3:33561)
0809 06:30:12.644075 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:12.644134 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:12.646202 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:12.646865 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:12.646874 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:12.646962 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:12.647535 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.647552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.648755 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.648758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.650050 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.650067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.650836 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.651033 (+ 197us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.651274 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.652036 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:30:12.652039 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:12.652077 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:30:12.652153 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:12.652515 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:12.652523 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:12.652529 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 06:30:12.652674 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 06:30:12.653156 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":342,"threads_started":1}
I20260809 06:30:12.655947 3297 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38855
I20260809 06:30:12.656294 3297 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:12.657266 3297 heartbeater.cc:511] Master 127.8.131.62:38855 requested a full tablet report, sending...
I20260809 06:30:12.658737 2644 ts_manager.cc:194] Registered new tserver with Master: b7ffadc0f66f4f5db30d808a8363f0df (127.8.131.3:45403)
I20260809 06:30:12.659567 2644 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.3:33561
I20260809 06:30:12.670253 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:30:12.681294 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.674757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.8.131.62:38855)
0809 06:30:12.675023 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:12.675032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:12.675563 (+ 531us) server_negotiation.cc:407] Connection header received
0809 06:30:12.675824 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:12.675830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:12.675945 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:12.676054 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:12.676578 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.677347 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.678835 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.679280 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.679432 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.680548 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:12.680560 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:12.680564 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:12.680582 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:12.680613 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:12.680625 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:12.680722 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:12.680901 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 06:30:12.680998 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 06:30:12.685367 2644 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:30:12.692317 2644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:30:12.701020 2916 heartbeater.cc:503] Master 127.8.131.62:38855 was elected leader, sending a full tablet report...
I20260809 06:30:12.716826 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.705499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.8.131.1:46565)
0809 06:30:12.706069 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:12.706078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:12.708025 (+ 1947us) server_negotiation.cc:407] Connection header received
0809 06:30:12.708121 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:12.708128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:12.708373 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:12.708563 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:12.709383 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.711087 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.714654 (+ 3567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.715608 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.715797 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.716098 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:12.716299 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 06:30:12.716486 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":495,"threads_started":1}
I20260809 06:30:12.716892 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.704959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.0.0.1:37400)
0809 06:30:12.707500 (+ 2541us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:12.707570 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:12.707793 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:12.708599 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:12.708609 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:12.708645 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:12.709164 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.709178 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.711231 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.711237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.714461 (+ 3224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.714476 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.715822 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.715828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.716034 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.716094 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 06:30:12.716256 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 06:30:12.716442 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2369,"mutex_wait_us":470,"spinlock_wait_cycles":10112,"thread_start_us":2178,"threads_started":1}
I20260809 06:30:12.717797 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.705311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:39343 (local address 127.0.0.1:46606)
0809 06:30:12.707982 (+ 2671us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:12.708001 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:12.708146 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:12.708932 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:12.708941 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:12.708985 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:12.709518 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.709527 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.711305 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.711312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.714100 (+ 2788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.714114 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.716784 (+ 2670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.716791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.717062 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.717114 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 06:30:12.717212 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 06:30:12.717357 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2538,"mutex_wait_us":387,"thread_start_us":2416,"threads_started":1}
I20260809 06:30:12.718138 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.705850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46606 (local address 127.8.131.2:39343)
0809 06:30:12.706603 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:12.706615 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:12.708109 (+ 1494us) server_negotiation.cc:407] Connection header received
0809 06:30:12.708293 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:12.708299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:12.708465 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:12.708690 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:12.709681 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.711106 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.714479 (+ 3373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.716186 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.716344 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.716606 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:12.717286 (+ 680us) server_negotiation.cc:299] Negotiation successful
0809 06:30:12.717471 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":430,"threads_started":1}
I20260809 06:30:12.718187 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.704933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38172 (local address 127.8.131.3:45403)
0809 06:30:12.706068 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:12.706078 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:12.708014 (+ 1936us) server_negotiation.cc:407] Connection header received
0809 06:30:12.708119 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:12.708127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:12.708369 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:12.708563 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:12.709778 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.711324 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.714198 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.715962 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.716129 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.716470 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:12.717521 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0809 06:30:12.717713 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":977,"thread_start_us":811,"threads_started":1}
I20260809 06:30:12.718426 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.704575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:45403 (local address 127.0.0.1:38172)
0809 06:30:12.707500 (+ 2925us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:12.707570 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:12.707793 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:12.708844 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:12.708850 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:12.708878 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:12.709488 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.709504 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.711454 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.711460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.713912 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.713930 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.716645 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.716656 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.716948 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.717041 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 06:30:12.717150 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 06:30:12.717340 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2669,"mutex_wait_us":346,"spinlock_wait_cycles":9216,"thread_start_us":982,"threads_started":1}
I20260809 06:30:12.721737 2821 tablet_service.cc:1467] Processing CreateTablet for tablet 2d761fd5190c4e2f8aff689fa8f4cd45 (DEFAULT_TABLE table=test-table [id=f79eaaff8cdc4b13b66ac1c3edebd13b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:12.722124 3201 tablet_service.cc:1467] Processing CreateTablet for tablet 2d761fd5190c4e2f8aff689fa8f4cd45 (DEFAULT_TABLE table=test-table [id=f79eaaff8cdc4b13b66ac1c3edebd13b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:12.722229 3007 tablet_service.cc:1467] Processing CreateTablet for tablet 2d761fd5190c4e2f8aff689fa8f4cd45 (DEFAULT_TABLE table=test-table [id=f79eaaff8cdc4b13b66ac1c3edebd13b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:12.723294 2821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d761fd5190c4e2f8aff689fa8f4cd45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:12.723483 3201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d761fd5190c4e2f8aff689fa8f4cd45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:12.723624 3007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d761fd5190c4e2f8aff689fa8f4cd45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:12.736761 3326 tablet_bootstrap.cc:492] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: Bootstrap starting.
I20260809 06:30:12.741540 3326 tablet_bootstrap.cc:654] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:12.741618 3328 tablet_bootstrap.cc:492] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: Bootstrap starting.
I20260809 06:30:12.742746 3326 log.cc:824] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:12.745491 3327 tablet_bootstrap.cc:492] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2: Bootstrap starting.
I20260809 06:30:12.745929 3328 tablet_bootstrap.cc:654] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:12.746917 3328 log.cc:824] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:12.749977 3327 tablet_bootstrap.cc:654] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:12.750571 3328 tablet_bootstrap.cc:492] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: No bootstrap required, opened a new log
I20260809 06:30:12.750865 3328 ts_tablet_manager.cc:1397] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260809 06:30:12.751194 3327 log.cc:824] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:12.753360 3326 tablet_bootstrap.cc:492] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: No bootstrap required, opened a new log
I20260809 06:30:12.753692 3326 ts_tablet_manager.cc:1397] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.007s
I20260809 06:30:12.754624 3327 tablet_bootstrap.cc:492] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2: No bootstrap required, opened a new log
I20260809 06:30:12.754923 3327 ts_tablet_manager.cc:1397] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260809 06:30:12.758831 3328 raft_consensus.cc:348] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } }
I20260809 06:30:12.759292 3328 raft_consensus.cc:374] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:12.759395 3328 raft_consensus.cc:729] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ffadc0f66f4f5db30d808a8363f0df, State: Initialized, Role: FOLLOWER
I20260809 06:30:12.760089 3328 consensus_queue.cc:260] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } }
I20260809 06:30:12.761140 3297 heartbeater.cc:503] Master 127.8.131.62:38855 was elected leader, sending a full tablet report...
I20260809 06:30:12.763022 3328 ts_tablet_manager.cc:1428] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: Time spent starting tablet: real 0.012s user 0.007s sys 0.003s
I20260809 06:30:12.766072 3326 raft_consensus.cc:348] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } }
I20260809 06:30:12.766620 3326 raft_consensus.cc:374] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:12.766752 3326 raft_consensus.cc:729] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a2546b1d70437e93bf65a97043ba65, State: Initialized, Role: FOLLOWER
I20260809 06:30:12.767230 3327 raft_consensus.cc:348] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } }
I20260809 06:30:12.767470 3326 consensus_queue.cc:260] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } }
I20260809 06:30:12.767750 3327 raft_consensus.cc:374] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:12.767874 3327 raft_consensus.cc:729] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5933e5225d15460e829f9fe68c4453c2, State: Initialized, Role: FOLLOWER
I20260809 06:30:12.768752 3107 heartbeater.cc:503] Master 127.8.131.62:38855 was elected leader, sending a full tablet report...
I20260809 06:30:12.768594 3327 consensus_queue.cc:260] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } }
I20260809 06:30:12.769698 3326 ts_tablet_manager.cc:1428] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260809 06:30:12.770705 3327 ts_tablet_manager.cc:1428] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2: Time spent starting tablet: real 0.016s user 0.016s sys 0.001s
W20260809 06:30:12.892300 3298 tablet.cc:2369] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:30:12.910970 3109 tablet.cc:2369] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:30:12.927812 2917 tablet.cc:2369] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:30:12.934104 3334 raft_consensus.cc:482] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:30:12.934429 3334 raft_consensus.cc:504] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } }
I20260809 06:30:12.936194 3334 leader_election.cc:288] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5933e5225d15460e829f9fe68c4453c2 (127.8.131.1:46565), 44a2546b1d70437e93bf65a97043ba65 (127.8.131.2:39343)
I20260809 06:30:12.942487 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.936634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:49031 (local address 127.8.131.1:46565)
0809 06:30:12.936833 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:12.936839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:12.937023 (+ 184us) server_negotiation.cc:407] Connection header received
0809 06:30:12.937238 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:12.937242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:12.937329 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:12.937435 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:12.937923 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.938904 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.940768 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.941583 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.941738 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.941838 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:12.942096 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 06:30:12.942221 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:30:12.942519 3299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.936493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.8.131.3:49031)
0809 06:30:12.936850 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:12.936870 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:12.937058 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:12.937492 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:12.937497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:12.937526 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:12.937751 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.937764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.939041 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.939046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.940595 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.940610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.941746 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.941752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.941921 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.941945 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 06:30:12.942055 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 06:30:12.942199 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":40}
I20260809 06:30:12.942922 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.936870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:57943 (local address 127.8.131.2:39343)
0809 06:30:12.937110 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:12.937118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:12.937683 (+ 565us) server_negotiation.cc:407] Connection header received
0809 06:30:12.937804 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:12.937811 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:12.937909 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:12.937990 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:12.938587 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.939460 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.941279 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.942013 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.942145 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.942236 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:12.942538 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 06:30:12.942658 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 06:30:12.943009 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:12.936669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:39343 (local address 127.8.131.3:57943)
0809 06:30:12.937579 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:12.937593 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:12.937702 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:12.938042 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:12.938046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:12.938071 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:12.938421 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.938433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.939608 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.939613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.941079 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:12.941092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:12.942187 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:12.942193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:12.942365 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:12.942401 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 06:30:12.942516 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 06:30:12.942633 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":1,"thread_start_us":551,"threads_started":1}
I20260809 06:30:12.943941 2848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d761fd5190c4e2f8aff689fa8f4cd45" candidate_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5933e5225d15460e829f9fe68c4453c2" is_pre_election: true
I20260809 06:30:12.944010 3041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d761fd5190c4e2f8aff689fa8f4cd45" candidate_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44a2546b1d70437e93bf65a97043ba65" is_pre_election: true
I20260809 06:30:12.944474 2848 raft_consensus.cc:2454] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ffadc0f66f4f5db30d808a8363f0df in term 0.
I20260809 06:30:12.944518 3041 raft_consensus.cc:2454] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ffadc0f66f4f5db30d808a8363f0df in term 0.
I20260809 06:30:12.945199 3136 leader_election.cc:302] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5933e5225d15460e829f9fe68c4453c2, b7ffadc0f66f4f5db30d808a8363f0df; no voters:
I20260809 06:30:12.945647 3334 raft_consensus.cc:2781] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:30:12.945812 3334 raft_consensus.cc:482] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:30:12.945923 3334 raft_consensus.cc:3037] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:12.948457 3334 raft_consensus.cc:504] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } }
I20260809 06:30:12.949051 3334 leader_election.cc:288] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [CANDIDATE]: Term 1 election: Requested vote from peers 5933e5225d15460e829f9fe68c4453c2 (127.8.131.1:46565), 44a2546b1d70437e93bf65a97043ba65 (127.8.131.2:39343)
I20260809 06:30:12.949502 3041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d761fd5190c4e2f8aff689fa8f4cd45" candidate_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44a2546b1d70437e93bf65a97043ba65"
I20260809 06:30:12.949527 2848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d761fd5190c4e2f8aff689fa8f4cd45" candidate_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5933e5225d15460e829f9fe68c4453c2"
I20260809 06:30:12.949733 3041 raft_consensus.cc:3037] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:12.949738 2848 raft_consensus.cc:3037] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:12.952028 2848 raft_consensus.cc:2454] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ffadc0f66f4f5db30d808a8363f0df in term 1.
I20260809 06:30:12.952085 3041 raft_consensus.cc:2454] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ffadc0f66f4f5db30d808a8363f0df in term 1.
I20260809 06:30:12.952476 3136 leader_election.cc:302] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5933e5225d15460e829f9fe68c4453c2, b7ffadc0f66f4f5db30d808a8363f0df; no voters:
I20260809 06:30:12.952785 3334 raft_consensus.cc:2781] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:12.953256 3334 raft_consensus.cc:686] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 1 LEADER]: Becoming Leader. State: Replica: b7ffadc0f66f4f5db30d808a8363f0df, State: Running, Role: LEADER
I20260809 06:30:12.953800 3334 consensus_queue.cc:237] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } }
I20260809 06:30:12.958671 2644 catalog_manager.cc:5495] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df reported cstate change: term changed from 0 to 1, leader changed from <none> to b7ffadc0f66f4f5db30d808a8363f0df (127.8.131.3). New cstate: current_term: 1 leader_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ffadc0f66f4f5db30d808a8363f0df" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 45403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 } health_report { overall_health: UNKNOWN } } }
I20260809 06:30:13.014940 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.009973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38174 (local address 127.8.131.3:45403)
0809 06:30:13.010153 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:13.010157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:13.010176 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:30:13.010331 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:13.010335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:13.010425 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:13.010558 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:30:13.010921 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:13.011709 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:13.013057 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:13.013448 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:13.013539 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:13.013974 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:30:13.014465 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:30:13.014572 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:13.014689 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 06:30:13.014748 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 06:30:13.023078 2848 raft_consensus.cc:1261] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [term 1 FOLLOWER]: Refusing update from remote peer b7ffadc0f66f4f5db30d808a8363f0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:13.023077 3041 raft_consensus.cc:1261] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [term 1 FOLLOWER]: Refusing update from remote peer b7ffadc0f66f4f5db30d808a8363f0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:13.024075 3342 consensus_queue.cc:1035] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "44a2546b1d70437e93bf65a97043ba65" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:13.024530 3334 consensus_queue.cc:1035] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5933e5225d15460e829f9fe68c4453c2" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:13.036247 3350 mvcc.cc:205] Tried to move back new op lower bound from 7316508725333049344 to 7316508725062742016. Current Snapshot: MvccSnapshot[applied={T|T < 7316508725333049344}]
I20260809 06:30:13.050165 2644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:13.050450 2644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:13.054375 2644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c29e95c659b2475a8f27189c6c740f74: Sending DeleteTablet for 3 replicas of tablet 2d761fd5190c4e2f8aff689fa8f4cd45
I20260809 06:30:13.055439 3201 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d761fd5190c4e2f8aff689fa8f4cd45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:13 UTC) from {username='slave'} at 127.0.0.1:38172
I20260809 06:30:13.055589 3007 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d761fd5190c4e2f8aff689fa8f4cd45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:13 UTC) from {username='slave'} at 127.0.0.1:46606
I20260809 06:30:13.055608 2821 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d761fd5190c4e2f8aff689fa8f4cd45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:13 UTC) from {username='slave'} at 127.0.0.1:37400
I20260809 06:30:13.059854 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 2733
I20260809 06:30:13.064199 3366 tablet_replica.cc:330] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2: stopping tablet replica
I20260809 06:30:13.064772 3366 raft_consensus.cc:2229] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:13.065390 3366 raft_consensus.cc:2258] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:13.068248 3366 ts_tablet_manager.cc:1905] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 5933e5225d15460e829f9fe68c4453c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:13.075935 3365 tablet_replica.cc:330] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: stopping tablet replica
I20260809 06:30:13.076406 3365 raft_consensus.cc:2229] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:13.077024 3365 raft_consensus.cc:2258] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:13.079013 3363 tablet_replica.cc:330] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: stopping tablet replica
I20260809 06:30:13.079427 3363 raft_consensus.cc:2229] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:30:13.080142 3363 raft_consensus.cc:2258] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:13.080650 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.061269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37416 (local address 127.8.131.1:46565)
0809 06:30:13.061454 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:13.061461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:13.072259 (+ 10798us) server_negotiation.cc:407] Connection header received
0809 06:30:13.072770 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:13.072777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:13.072878 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:13.072950 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:13.073597 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:13.074726 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:13.076925 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:13.077533 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:13.077635 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:13.079199 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:13.079208 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:13.079221 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:13.079868 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:13.079910 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:13.079929 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:13.080064 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:13.080165 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 06:30:13.080270 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 06:30:13.081398 2909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:13.082720 3363 ts_tablet_manager.cc:1905] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:13.085873 3365 ts_tablet_manager.cc:1905] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:13.090230 3363 ts_tablet_manager.cc:1918] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:13.090415 3363 log.cc:1196] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/2d761fd5190c4e2f8aff689fa8f4cd45
I20260809 06:30:13.090983 3363 ts_tablet_manager.cc:1939] T 2d761fd5190c4e2f8aff689fa8f4cd45 P b7ffadc0f66f4f5db30d808a8363f0df: Deleting consensus metadata
I20260809 06:30:13.093824 2620 catalog_manager.cc:4841] TS b7ffadc0f66f4f5db30d808a8363f0df (127.8.131.3:45403): tablet 2d761fd5190c4e2f8aff689fa8f4cd45 (table test-table [id=f79eaaff8cdc4b13b66ac1c3edebd13b]) successfully deleted
I20260809 06:30:13.096033 3365 ts_tablet_manager.cc:1918] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:13.096333 3365 log.cc:1196] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/2d761fd5190c4e2f8aff689fa8f4cd45
I20260809 06:30:13.097066 3365 ts_tablet_manager.cc:1939] T 2d761fd5190c4e2f8aff689fa8f4cd45 P 44a2546b1d70437e93bf65a97043ba65: Deleting consensus metadata
I20260809 06:30:13.098879 2618 catalog_manager.cc:4841] TS 44a2546b1d70437e93bf65a97043ba65 (127.8.131.2:39343): tablet 2d761fd5190c4e2f8aff689fa8f4cd45 (table test-table [id=f79eaaff8cdc4b13b66ac1c3edebd13b]) successfully deleted
I20260809 06:30:13.193702 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 2733
W20260809 06:30:13.220968 2621 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.131.1:46565 (error 108)
I20260809 06:30:13.221834 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 2921
I20260809 06:30:13.223073 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.222277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.0.0.1:37418)
0809 06:30:13.222718 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:13.222868 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":210,"threads_started":1}
W20260809 06:30:13.223341 2621 catalog_manager.cc:4584] TS 5933e5225d15460e829f9fe68c4453c2 (127.8.131.1:46565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d761fd5190c4e2f8aff689fa8f4cd45: Network error: Client connection negotiation failed: client connection to 127.8.131.1:46565: connect: Connection refused (error 111)
I20260809 06:30:13.229986 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.222999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.8.131.2:39343)
0809 06:30:13.223186 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:13.223191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:13.223216 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:30:13.223284 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:13.223290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:13.223407 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:13.223521 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:13.224079 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:13.225238 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:13.226542 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:13.227207 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:13.227340 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:13.228378 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:13.228389 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:13.228406 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:13.229218 (+ 812us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:13.229278 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:13.229303 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:13.229420 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:13.229509 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 06:30:13.229618 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:30:13.230829 3100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:13.277027 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.276462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.0.0.1:37432)
0809 06:30:13.276702 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:13.276820 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 06:30:13.278293 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.277819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.0.0.1:37436)
0809 06:30:13.278003 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:13.278099 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 06:30:13.321570 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 2921
I20260809 06:30:13.338711 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.338245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.0.0.1:37444)
0809 06:30:13.338470 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:13.338556 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 06:30:13.339949 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.339464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.0.0.1:37456)
0809 06:30:13.339620 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:13.339771 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 06:30:13.353540 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 3111
I20260809 06:30:13.361650 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.354346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38182 (local address 127.8.131.3:45403)
0809 06:30:13.354528 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:13.354538 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:13.354664 (+ 126us) server_negotiation.cc:407] Connection header received
0809 06:30:13.354890 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:13.354894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:13.354975 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:13.355060 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:13.355613 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:13.356478 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:13.358464 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:13.358902 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:13.358987 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:13.360255 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:13.360264 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:13.360278 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:13.360858 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:13.360906 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:13.360925 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:13.361017 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:13.361192 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 06:30:13.361269 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 06:30:13.362406 3289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:13.420794 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.420217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.0.0.1:37460)
0809 06:30:13.420487 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:13.420604 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260809 06:30:13.422084 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.421597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.0.0.1:37476)
0809 06:30:13.421828 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:13.421925 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 06:30:13.483989 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 3111
I20260809 06:30:13.512017 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 2595
I20260809 06:30:13.513146 2693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:13.622133 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:13.621379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:46565 (local address 127.0.0.1:37486)
0809 06:30:13.621791 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:13.621921 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233}
I20260809 06:30:13.622151 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 2595
2026-08-09T06:30:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3037 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:13 dist-test-slave-mjjr krb5kdc[3377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:13 dist-test-slave-mjjr krb5kdc[3377](info): set up 2 sockets
Aug 09 06:30:13 dist-test-slave-mjjr krb5kdc[3377](info): commencing operation
krb5kdc: starting...
W20260809 06:30:14.939440 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:15 dist-test-slave-mjjr krb5kdc[3377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:30:15.231242 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38389
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:38389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:15.585388 3394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:15.585688 3394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:15.585901 3394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:15.593336 3394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:15.593443 3394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:15.593497 3394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:15.593684 3394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:15.593783 3394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:15.603636 3394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:38389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:38389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:15.605016 3394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:15.606323 3394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:15.618274 3400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:15.618492 3404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:15.618546 3402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:15.619218 3394 server_base.cc:1034] running on GCE node
I20260809 06:30:15.620045 3394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:15.621435 3394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:15.622668 3394 hybrid_clock.cc:648] HybridClock initialized: now 1786257015622605 us; error 49 us; skew 500 ppm
I20260809 06:30:15.623059 3394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:15 dist-test-slave-mjjr krb5kdc[3377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257015, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:15.631081 3394 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:30:15.632655 3394 webserver.cc:460] Webserver started at http://127.8.131.62:35243/ using document root <none> and password file <none>
I20260809 06:30:15.633368 3394 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:15.633512 3394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:15.633833 3394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:15.637768 3394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "f571446f1b634de289af27bf1fbd4901"
format_stamp: "Formatted at 2026-08-09 06:30:15 on dist-test-slave-mjjr"
server_key: "301475d28c571818e839af97eb6cca21"
server_key_iv: "a1c0fba1a849a3fc78319f63aec4c6b1"
server_key_version: "encryptionkey@0"
I20260809 06:30:15.638732 3394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "f571446f1b634de289af27bf1fbd4901"
format_stamp: "Formatted at 2026-08-09 06:30:15 on dist-test-slave-mjjr"
server_key: "301475d28c571818e839af97eb6cca21"
server_key_iv: "a1c0fba1a849a3fc78319f63aec4c6b1"
server_key_version: "encryptionkey@0"
I20260809 06:30:15.644055 3394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 06:30:15.647536 3414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:15.648818 3394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:15.648996 3394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "f571446f1b634de289af27bf1fbd4901"
format_stamp: "Formatted at 2026-08-09 06:30:15 on dist-test-slave-mjjr"
server_key: "301475d28c571818e839af97eb6cca21"
server_key_iv: "a1c0fba1a849a3fc78319f63aec4c6b1"
server_key_version: "encryptionkey@0"
I20260809 06:30:15.649188 3394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:15.671093 3394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:15.675084 3394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:15.675390 3394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:15.694595 3394 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:38389
I20260809 06:30:15.694628 3487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:38389 every 8 connection(s)
I20260809 06:30:15.696008 3394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:15.698542 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 3394
I20260809 06:30:15.698756 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:15.699368 8716 external_mini_cluster.cc:1428] Setting key 1a3e5ff8a67d3232c21385bdc146e00b
I20260809 06:30:15.700409 3488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:15.712808 3488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901: Bootstrap starting.
Aug 09 06:30:15 dist-test-slave-mjjr krb5kdc[3377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:15.716372 3488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:15.717470 3488 log.cc:824] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:15.720928 3488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901: No bootstrap required, opened a new log
I20260809 06:30:15.722472 3491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:15.701308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.8.131.62:38389)
0809 06:30:15.702118 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:15.702129 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:15.702169 (+ 40us) server_negotiation.cc:407] Connection header received
0809 06:30:15.703870 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:15.703892 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:15.704260 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:15.704686 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:15.706338 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:15.707745 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:15.709084 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:15.709798 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:15.710058 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:15.714459 (+ 4401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:15.714466 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:15.714479 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:15.714522 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:15.717724 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:15.718627 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:15.718633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:15.718640 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:15.718859 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:15.719287 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:15.719292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:15.719295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:15.720006 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:15.720317 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:15.720702 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 06:30:15.721138 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":356,"threads_started":1}
I20260809 06:30:15.728965 3488 raft_consensus.cc:348] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571446f1b634de289af27bf1fbd4901" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38389 } }
I20260809 06:30:15.729293 3488 raft_consensus.cc:374] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:15.729368 3488 raft_consensus.cc:729] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f571446f1b634de289af27bf1fbd4901, State: Initialized, Role: FOLLOWER
I20260809 06:30:15.730002 3488 consensus_queue.cc:260] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571446f1b634de289af27bf1fbd4901" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38389 } }
I20260809 06:30:15.730273 3488 raft_consensus.cc:388] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:15.730441 3488 raft_consensus.cc:482] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:15.730628 3488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:15.732812 3488 raft_consensus.cc:504] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571446f1b634de289af27bf1fbd4901" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38389 } }
I20260809 06:30:15.733250 3488 leader_election.cc:302] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f571446f1b634de289af27bf1fbd4901; no voters:
I20260809 06:30:15.733778 3488 leader_election.cc:288] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:15.733961 3493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:15.734714 3493 raft_consensus.cc:686] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [term 1 LEADER]: Becoming Leader. State: Replica: f571446f1b634de289af27bf1fbd4901, State: Running, Role: LEADER
I20260809 06:30:15.735347 3493 consensus_queue.cc:237] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571446f1b634de289af27bf1fbd4901" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38389 } }
I20260809 06:30:15.736135 3488 sys_catalog.cc:564] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:15.738440 3497 sys_catalog.cc:455] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f571446f1b634de289af27bf1fbd4901" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571446f1b634de289af27bf1fbd4901" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38389 } } }
I20260809 06:30:15.738482 3499 sys_catalog.cc:455] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f571446f1b634de289af27bf1fbd4901. Latest consensus state: current_term: 1 leader_uuid: "f571446f1b634de289af27bf1fbd4901" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571446f1b634de289af27bf1fbd4901" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 38389 } } }
I20260809 06:30:15.738832 3497 sys_catalog.cc:458] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:15.738909 3499 sys_catalog.cc:458] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:15.741003 3506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:15.747015 3506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:15.755367 3506 catalog_manager.cc:1347] Generated new cluster ID: 5b32c1850ded475d859993944368fc68
I20260809 06:30:15.755491 3506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:15.765024 3506 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:15.766863 3506 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:15.777172 3506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901: Generated new TSK 0
I20260809 06:30:15.778146 3506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:30:15.852983 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38389
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:16.214819 3529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:16.215075 3529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:16.215171 3529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:16.222208 3529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:16.222364 3529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:16.222536 3529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:30:16.232179 3529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:16.233485 3529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:16.234782 3529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:16.248229 3535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:16.248261 3539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:16.248356 3542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:16.249531 3529 server_base.cc:1034] running on GCE node
I20260809 06:30:16.250120 3529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:16.251026 3529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:16.252283 3529 hybrid_clock.cc:648] HybridClock initialized: now 1786257016252218 us; error 62 us; skew 500 ppm
I20260809 06:30:16.252547 3529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:16 dist-test-slave-mjjr krb5kdc[3377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257016, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:16.255769 3529 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:30:16.257221 3529 webserver.cc:460] Webserver started at http://127.8.131.1:40781/ using document root <none> and password file <none>
I20260809 06:30:16.257840 3529 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:16.257946 3529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:16.258201 3529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:16.261014 3529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "be09ecfd302b442dbbf45943b0a0f1df"
format_stamp: "Formatted at 2026-08-09 06:30:16 on dist-test-slave-mjjr"
server_key: "6f5fdb6fcf5380fcb8494e8304737670"
server_key_iv: "986b9e6f508d957e13d17528475eee9b"
server_key_version: "encryptionkey@0"
I20260809 06:30:16.261766 3529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "be09ecfd302b442dbbf45943b0a0f1df"
format_stamp: "Formatted at 2026-08-09 06:30:16 on dist-test-slave-mjjr"
server_key: "6f5fdb6fcf5380fcb8494e8304737670"
server_key_iv: "986b9e6f508d957e13d17528475eee9b"
server_key_version: "encryptionkey@0"
I20260809 06:30:16.266710 3529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:30:16.270073 3552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:16.271330 3529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:30:16.271521 3529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "be09ecfd302b442dbbf45943b0a0f1df"
format_stamp: "Formatted at 2026-08-09 06:30:16 on dist-test-slave-mjjr"
server_key: "6f5fdb6fcf5380fcb8494e8304737670"
server_key_iv: "986b9e6f508d957e13d17528475eee9b"
server_key_version: "encryptionkey@0"
I20260809 06:30:16.271790 3529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:16.283845 3529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:16.288013 3529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:16.288297 3529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:16.289222 3529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:16.290496 3529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:16.290597 3529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:16.290736 3529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:16.290807 3529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:16.324333 3529 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:35805
I20260809 06:30:16.324393 3708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:35805 every 8 connection(s)
I20260809 06:30:16.325623 3529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:30:16.326083 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 3529
I20260809 06:30:16.326278 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:30:16.326954 8716 external_mini_cluster.cc:1428] Setting key 4575f145e579aad6926364a92e595c5a
Aug 09 06:30:16 dist-test-slave-mjjr krb5kdc[3377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257016, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:16.350826 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:16.328352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:57933 (local address 127.8.131.62:38389)
0809 06:30:16.328881 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:16.328887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:16.329985 (+ 1098us) server_negotiation.cc:407] Connection header received
0809 06:30:16.332876 (+ 2891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:16.332880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:16.332940 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:16.333029 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:16.335259 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:16.336124 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:16.338752 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:16.339161 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:16.339257 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:16.343611 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:16.343618 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:16.343623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:16.343691 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:16.347421 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:16.348200 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:16.348208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:16.348214 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:16.348369 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:16.348775 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:16.348783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:16.348789 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:16.349187 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:16.349340 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:16.350022 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 06:30:16.350306 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":261,"threads_started":1}
I20260809 06:30:16.351405 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:16.328906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38389 (local address 127.8.131.1:57933)
0809 06:30:16.329792 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:16.329846 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:16.332585 (+ 2739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:16.333317 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:16.333328 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:16.334114 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:16.334994 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:16.335022 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:16.336328 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:16.336336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:16.338590 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:16.338609 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:16.339394 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:16.339403 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:16.339755 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:16.340502 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:30:16.340536 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:16.343382 (+ 2846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:16.347560 (+ 4178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:16.347566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:16.347578 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:16.348073 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:16.348480 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:16.348483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:16.348485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:16.348657 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:16.349307 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:16.349313 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:16.349626 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 06:30:16.349847 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 06:30:16.350246 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":352,"threads_started":1}
I20260809 06:30:16.352985 3710 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38389
I20260809 06:30:16.353339 3710 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:16.354275 3710 heartbeater.cc:511] Master 127.8.131.62:38389 requested a full tablet report, sending...
I20260809 06:30:16.356518 3436 ts_manager.cc:194] Registered new tserver with Master: be09ecfd302b442dbbf45943b0a0f1df (127.8.131.1:35805)
I20260809 06:30:16.357988 3436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:57933
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:30:16.394863 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38389
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:16.735255 3718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:16.735571 3718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:16.735774 3718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:16.743620 3718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:16.743853 3718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:16.744032 3718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:30:16.753806 3718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:16.755075 3718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:16.756392 3718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:16.769364 3726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:16.769429 3730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:16.769358 3727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:16.770979 3718 server_base.cc:1034] running on GCE node
I20260809 06:30:16.771625 3718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:16.772655 3718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:16.773865 3718 hybrid_clock.cc:648] HybridClock initialized: now 1786257016773819 us; error 27 us; skew 500 ppm
I20260809 06:30:16.774221 3718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:16 dist-test-slave-mjjr krb5kdc[3377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257016, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:16.777750 3718 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:30:16.779178 3718 webserver.cc:460] Webserver started at http://127.8.131.2:41799/ using document root <none> and password file <none>
I20260809 06:30:16.779980 3718 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:16.780121 3718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:16.780449 3718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:16.783404 3718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "9db63456ddd4468f9c6087c31b90eaf2"
format_stamp: "Formatted at 2026-08-09 06:30:16 on dist-test-slave-mjjr"
server_key: "013c015313f6b2418a4a5789f561d1d7"
server_key_iv: "693bc52df2213503488b366edb0b2479"
server_key_version: "encryptionkey@0"
I20260809 06:30:16.784281 3718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "9db63456ddd4468f9c6087c31b90eaf2"
format_stamp: "Formatted at 2026-08-09 06:30:16 on dist-test-slave-mjjr"
server_key: "013c015313f6b2418a4a5789f561d1d7"
server_key_iv: "693bc52df2213503488b366edb0b2479"
server_key_version: "encryptionkey@0"
I20260809 06:30:16.789261 3718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:30:16.792641 3740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:16.793985 3718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:30:16.794174 3718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "9db63456ddd4468f9c6087c31b90eaf2"
format_stamp: "Formatted at 2026-08-09 06:30:16 on dist-test-slave-mjjr"
server_key: "013c015313f6b2418a4a5789f561d1d7"
server_key_iv: "693bc52df2213503488b366edb0b2479"
server_key_version: "encryptionkey@0"
I20260809 06:30:16.794380 3718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:16.808058 3718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:16.812006 3718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:16.812318 3718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:16.813261 3718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:16.814558 3718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:16.814663 3718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:16.814833 3718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:16.814913 3718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:16.848932 3718 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:39733
I20260809 06:30:16.848984 3890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:39733 every 8 connection(s)
I20260809 06:30:16.850239 3718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:30:16.850888 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 3718
I20260809 06:30:16.851086 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:30:16.851536 8716 external_mini_cluster.cc:1428] Setting key 2b162b7939dc986ba0607da3df4bfbfd
Aug 09 06:30:16 dist-test-slave-mjjr krb5kdc[3377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257016, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:16.872030 3897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:16.853458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:52499 (local address 127.8.131.62:38389)
0809 06:30:16.853951 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:16.853956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:16.855144 (+ 1188us) server_negotiation.cc:407] Connection header received
0809 06:30:16.857125 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:16.857134 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:16.857221 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:16.857372 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:16.858834 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:16.860028 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:16.861511 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:16.862286 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:16.862433 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:16.865917 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:16.865926 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:16.865933 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:16.865979 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:16.868621 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:16.869583 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:16.869587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:16.869589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:16.869718 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:16.870204 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:16.870207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:16.870208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:16.870459 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:16.870567 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:16.871490 (+ 923us) server_negotiation.cc:299] Negotiation successful
0809 06:30:16.871757 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":266,"threads_started":1}
I20260809 06:30:16.873581 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:16.854103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38389 (local address 127.8.131.2:52499)
0809 06:30:16.854938 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:16.854993 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:16.856850 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:16.857553 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:16.857561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:16.858110 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:16.858661 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:16.858679 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:16.860169 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:16.860173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:16.861366 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:16.861378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:16.862488 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:16.862493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:16.862673 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:16.863372 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:30:16.863392 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:16.865687 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:16.868765 (+ 3078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:16.868776 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:16.868793 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:16.869462 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:16.869826 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:16.869834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:16.869840 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:16.870096 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:16.870601 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:16.870613 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:16.871056 (+ 443us) client_negotiation.cc:769] Sending connection context
0809 06:30:16.871371 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 06:30:16.871951 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":342,"threads_started":1}
I20260809 06:30:16.875090 3895 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38389
I20260809 06:30:16.875391 3895 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:16.876222 3895 heartbeater.cc:511] Master 127.8.131.62:38389 requested a full tablet report, sending...
I20260809 06:30:16.877789 3436 ts_manager.cc:194] Registered new tserver with Master: 9db63456ddd4468f9c6087c31b90eaf2 (127.8.131.2:39733)
I20260809 06:30:16.878621 3436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:52499
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:30:16.919273 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38389
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:43183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:17.253948 3903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:17.254236 3903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:17.254398 3903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:17.261549 3903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:17.261679 3903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:17.261844 3903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:30:17.271201 3903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:43183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:38389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:17.272446 3903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:17.273717 3903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:17.286971 3912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:17.286998 3913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:17.287070 3915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:17.287832 3903 server_base.cc:1034] running on GCE node
I20260809 06:30:17.288406 3903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:17.289276 3903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:17.290500 3903 hybrid_clock.cc:648] HybridClock initialized: now 1786257017290467 us; error 57 us; skew 500 ppm
I20260809 06:30:17.290797 3903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:17 dist-test-slave-mjjr krb5kdc[3377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257017, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:17.293932 3903 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:30:17.295329 3903 webserver.cc:460] Webserver started at http://127.8.131.3:32997/ using document root <none> and password file <none>
I20260809 06:30:17.296052 3903 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:17.296198 3903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:17.296512 3903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:17.299489 3903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "55fa44bd0b67461c8af9c5589e61b7e1"
format_stamp: "Formatted at 2026-08-09 06:30:17 on dist-test-slave-mjjr"
server_key: "dedc51de1d50c39db67a709c41679496"
server_key_iv: "07d7e4b89f7c5a6b27d4de9708c0806d"
server_key_version: "encryptionkey@0"
I20260809 06:30:17.300309 3903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "55fa44bd0b67461c8af9c5589e61b7e1"
format_stamp: "Formatted at 2026-08-09 06:30:17 on dist-test-slave-mjjr"
server_key: "dedc51de1d50c39db67a709c41679496"
server_key_iv: "07d7e4b89f7c5a6b27d4de9708c0806d"
server_key_version: "encryptionkey@0"
I20260809 06:30:17.305171 3903 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 06:30:17.308522 3922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:17.309756 3903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 06:30:17.309937 3903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "55fa44bd0b67461c8af9c5589e61b7e1"
format_stamp: "Formatted at 2026-08-09 06:30:17 on dist-test-slave-mjjr"
server_key: "dedc51de1d50c39db67a709c41679496"
server_key_iv: "07d7e4b89f7c5a6b27d4de9708c0806d"
server_key_version: "encryptionkey@0"
I20260809 06:30:17.310135 3903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:17.321630 3903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:17.326268 3903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:17.326601 3903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:17.327770 3903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:17.329380 3903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:17.329474 3903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:17.329600 3903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:17.329670 3903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:17.361655 3710 heartbeater.cc:503] Master 127.8.131.62:38389 was elected leader, sending a full tablet report...
I20260809 06:30:17.365047 3903 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:37023
I20260809 06:30:17.365137 4077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:37023 every 8 connection(s)
I20260809 06:30:17.366276 3903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:30:17.367486 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 3903
I20260809 06:30:17.367704 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:30:17.368302 8716 external_mini_cluster.cc:1428] Setting key f4f67bf4377ae9b79c505ab66b4dbebc
Aug 09 06:30:17 dist-test-slave-mjjr krb5kdc[3377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257017, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:17.386690 3897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.369100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:47351 (local address 127.8.131.62:38389)
0809 06:30:17.369273 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.369277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.370624 (+ 1347us) server_negotiation.cc:407] Connection header received
0809 06:30:17.372869 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.372873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.372937 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.373018 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:17.374507 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.375257 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.376971 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.377373 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.377468 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.380785 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:17.380789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:17.380792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:17.380822 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:17.383793 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.384485 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.384490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.384493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.384611 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.385026 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.385030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.385034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.385299 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:17.385459 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.386106 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.386355 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:30:17.387511 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.369572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:38389 (local address 127.8.131.3:47351)
0809 06:30:17.370438 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:17.370501 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:17.372620 (+ 2119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:17.373219 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:17.373226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:17.373776 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:17.374337 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.374353 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.375442 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.375445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.376837 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.376850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.377557 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.377562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.377747 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.378361 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:30:17.378382 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:17.380581 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:17.383929 (+ 3348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:17.383937 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:17.383952 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:17.384362 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:17.384717 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:17.384722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:17.384726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:17.384906 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:17.385430 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:17.385437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:17.385741 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 06:30:17.385992 (+ 251us) client_negotiation.cc:240] Negotiation successful
0809 06:30:17.386502 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":361,"threads_started":1}
I20260809 06:30:17.388789 4078 heartbeater.cc:347] Connected to a master server at 127.8.131.62:38389
I20260809 06:30:17.389061 4078 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:17.389703 4078 heartbeater.cc:511] Master 127.8.131.62:38389 requested a full tablet report, sending...
I20260809 06:30:17.390988 3436 ts_manager.cc:194] Registered new tserver with Master: 55fa44bd0b67461c8af9c5589e61b7e1 (127.8.131.3:37023)
I20260809 06:30:17.391803 3436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:47351
I20260809 06:30:17.395771 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:30:17.413058 3897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.399770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48940 (local address 127.8.131.62:38389)
0809 06:30:17.399946 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.399972 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.400418 (+ 446us) server_negotiation.cc:407] Connection header received
0809 06:30:17.400644 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.400649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.400728 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.400829 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:17.402284 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.402984 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.404428 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.404860 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.405008 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.407755 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:17.407768 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:17.407772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:17.407802 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:17.410101 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.410874 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.410879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.410882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.411011 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.411513 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.411517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.411519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.411848 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:17.412153 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.412570 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.412768 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 06:30:17.416988 3436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:30:17.423804 3436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:30:17.458087 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.435786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36324 (local address 127.8.131.2:39733)
0809 06:30:17.448491 (+ 12705us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.448499 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.448529 (+ 30us) server_negotiation.cc:407] Connection header received
0809 06:30:17.448638 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.448647 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.448908 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.450057 (+ 1149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:17.450328 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.452422 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.455468 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.456729 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.456882 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.457080 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.457572 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.457727 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12235,"thread_start_us":12085,"threads_started":1}
I20260809 06:30:17.458226 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.436256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:37023 (local address 127.0.0.1:40662)
0809 06:30:17.448552 (+ 12296us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:17.448573 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:17.448708 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:17.449520 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:17.449528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:17.449573 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:17.450155 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.450171 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.453043 (+ 2872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.453052 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.455559 (+ 2507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.455571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.457254 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.457262 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.457497 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.457530 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 06:30:17.457619 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 06:30:17.457746 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12144,"mutex_wait_us":113,"thread_start_us":958,"threads_started":1}
I20260809 06:30:17.458267 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.435553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:39733 (local address 127.0.0.1:36324)
0809 06:30:17.436924 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:17.436964 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:17.437135 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:17.449154 (+ 12019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:17.449161 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:17.449192 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:17.449745 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.449765 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.452593 (+ 2828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.452601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.455282 (+ 2681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.455296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.456951 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.456971 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.457281 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.457366 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 06:30:17.457521 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 06:30:17.457721 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"mutex_wait_us":91,"thread_start_us":12705,"threads_started":1}
I20260809 06:30:17.459311 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.436898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.8.131.3:37023)
0809 06:30:17.448247 (+ 11349us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.448256 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.448850 (+ 594us) server_negotiation.cc:407] Connection header received
0809 06:30:17.448959 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.448967 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.449183 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.450386 (+ 1203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:17.450732 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.452847 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.455735 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.456863 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.456989 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.457228 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.458801 (+ 1573us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.458981 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11187,"thread_start_us":11352,"threads_started":1}
I20260809 06:30:17.461584 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.449410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35805 (local address 127.0.0.1:38196)
0809 06:30:17.453120 (+ 3710us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:17.453159 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:17.453286 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:17.453973 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:17.453977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:17.454010 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:17.454605 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.454617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.456317 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.456323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.458797 (+ 2474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.458811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.460773 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.460781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.460965 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.460995 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 06:30:17.461104 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 06:30:17.461215 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3574,"mutex_wait_us":25,"thread_start_us":3395,"threads_started":1}
I20260809 06:30:17.461669 4099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.448971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.8.131.1:35805)
0809 06:30:17.452782 (+ 3811us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.452792 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.453418 (+ 626us) server_negotiation.cc:407] Connection header received
0809 06:30:17.453520 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.453529 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.453745 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.454765 (+ 1020us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:17.454932 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.456151 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.459016 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.460523 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.460693 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.460922 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.461173 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.461359 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3623,"thread_start_us":3452,"threads_started":1}
I20260809 06:30:17.462610 3805 tablet_service.cc:1467] Processing CreateTablet for tablet c1de250695b44403bae83d78181e0ebf (DEFAULT_TABLE table=test-table [id=67fb2f98e4c54f5eb647207bdbe659e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:17.462946 3995 tablet_service.cc:1467] Processing CreateTablet for tablet c1de250695b44403bae83d78181e0ebf (DEFAULT_TABLE table=test-table [id=67fb2f98e4c54f5eb647207bdbe659e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:17.464184 3995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1de250695b44403bae83d78181e0ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:17.464213 3805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1de250695b44403bae83d78181e0ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:17.465075 3616 tablet_service.cc:1467] Processing CreateTablet for tablet c1de250695b44403bae83d78181e0ebf (DEFAULT_TABLE table=test-table [id=67fb2f98e4c54f5eb647207bdbe659e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:17.466429 3616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1de250695b44403bae83d78181e0ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:17.474835 4101 tablet_bootstrap.cc:492] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: Bootstrap starting.
I20260809 06:30:17.478109 4102 tablet_bootstrap.cc:492] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: Bootstrap starting.
I20260809 06:30:17.478866 4101 tablet_bootstrap.cc:654] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:17.480346 4103 tablet_bootstrap.cc:492] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: Bootstrap starting.
I20260809 06:30:17.480188 4101 log.cc:824] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:17.481550 4102 tablet_bootstrap.cc:654] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:17.482561 4102 log.cc:824] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:17.483680 4103 tablet_bootstrap.cc:654] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:17.484424 4101 tablet_bootstrap.cc:492] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: No bootstrap required, opened a new log
I20260809 06:30:17.484702 4101 ts_tablet_manager.cc:1397] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260809 06:30:17.484745 4103 log.cc:824] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:17.485584 4102 tablet_bootstrap.cc:492] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: No bootstrap required, opened a new log
I20260809 06:30:17.485836 4102 ts_tablet_manager.cc:1397] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260809 06:30:17.487198 4103 tablet_bootstrap.cc:492] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: No bootstrap required, opened a new log
I20260809 06:30:17.487447 4103 ts_tablet_manager.cc:1397] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 06:30:17.494238 4103 raft_consensus.cc:348] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } }
I20260809 06:30:17.494606 4103 raft_consensus.cc:374] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:17.494735 4103 raft_consensus.cc:729] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be09ecfd302b442dbbf45943b0a0f1df, State: Initialized, Role: FOLLOWER
I20260809 06:30:17.495222 4101 raft_consensus.cc:348] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } }
I20260809 06:30:17.495347 4103 consensus_queue.cc:260] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } }
I20260809 06:30:17.495656 4101 raft_consensus.cc:374] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:17.495796 4101 raft_consensus.cc:729] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55fa44bd0b67461c8af9c5589e61b7e1, State: Initialized, Role: FOLLOWER
I20260809 06:30:17.496524 4101 consensus_queue.cc:260] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } }
I20260809 06:30:17.497176 4103 ts_tablet_manager.cc:1428] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 06:30:17.498407 4078 heartbeater.cc:503] Master 127.8.131.62:38389 was elected leader, sending a full tablet report...
I20260809 06:30:17.498508 4102 raft_consensus.cc:348] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } }
I20260809 06:30:17.498939 4102 raft_consensus.cc:374] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:17.499053 4101 ts_tablet_manager.cc:1428] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260809 06:30:17.499037 4102 raft_consensus.cc:729] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db63456ddd4468f9c6087c31b90eaf2, State: Initialized, Role: FOLLOWER
I20260809 06:30:17.499966 4102 consensus_queue.cc:260] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } }
I20260809 06:30:17.501034 3895 heartbeater.cc:503] Master 127.8.131.62:38389 was elected leader, sending a full tablet report...
I20260809 06:30:17.501646 4102 ts_tablet_manager.cc:1428] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
I20260809 06:30:17.529066 4111 raft_consensus.cc:482] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:30:17.529345 4111 raft_consensus.cc:504] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } }
I20260809 06:30:17.530936 4111 leader_election.cc:288] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9db63456ddd4468f9c6087c31b90eaf2 (127.8.131.2:39733), be09ecfd302b442dbbf45943b0a0f1df (127.8.131.1:35805)
I20260809 06:30:17.537744 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.531321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:39733 (local address 127.8.131.3:38029)
0809 06:30:17.531559 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:17.531583 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:17.531768 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:17.532222 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:17.532229 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:17.532265 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:17.532491 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.532502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.533774 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.533777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.535591 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.535606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.537001 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.537007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.537193 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.537225 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 06:30:17.537336 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 06:30:17.537478 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 06:30:17.537777 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.531421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:38029 (local address 127.8.131.2:39733)
0809 06:30:17.531661 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.531666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.531685 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:30:17.531969 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.531976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.532075 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.532202 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:17.532662 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.533638 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.535750 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.536790 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.536924 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.537022 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.537364 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.537510 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:30:17.538653 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.531764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:35805 (local address 127.8.131.3:59977)
0809 06:30:17.532547 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:17.532575 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:17.532689 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:17.532996 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:17.532999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:17.533016 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:17.533295 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.533325 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.534377 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.534383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.536225 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:17.536237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.537975 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.537985 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:17.538175 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.538208 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 06:30:17.538315 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 06:30:17.538427 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":437,"threads_started":1}
I20260809 06:30:17.538766 4099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.531901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:59977 (local address 127.8.131.1:35805)
0809 06:30:17.532106 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.532113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.532665 (+ 552us) server_negotiation.cc:407] Connection header received
0809 06:30:17.532805 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.532811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.532881 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.532991 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:17.533459 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.534213 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.536367 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.537560 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.537689 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.537781 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.538386 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.538529 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 06:30:17.538846 3832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1de250695b44403bae83d78181e0ebf" candidate_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db63456ddd4468f9c6087c31b90eaf2" is_pre_election: true
I20260809 06:30:17.539389 3832 raft_consensus.cc:2454] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55fa44bd0b67461c8af9c5589e61b7e1 in term 0.
I20260809 06:30:17.539584 3647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1de250695b44403bae83d78181e0ebf" candidate_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be09ecfd302b442dbbf45943b0a0f1df" is_pre_election: true
I20260809 06:30:17.540135 3928 leader_election.cc:302] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55fa44bd0b67461c8af9c5589e61b7e1, 9db63456ddd4468f9c6087c31b90eaf2; no voters:
I20260809 06:30:17.540151 3647 raft_consensus.cc:2454] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55fa44bd0b67461c8af9c5589e61b7e1 in term 0.
I20260809 06:30:17.540532 4111 raft_consensus.cc:2781] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:30:17.540711 4111 raft_consensus.cc:482] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:30:17.540820 4111 raft_consensus.cc:3037] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:17.543133 4111 raft_consensus.cc:504] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } }
I20260809 06:30:17.543759 4111 leader_election.cc:288] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [CANDIDATE]: Term 1 election: Requested vote from peers 9db63456ddd4468f9c6087c31b90eaf2 (127.8.131.2:39733), be09ecfd302b442dbbf45943b0a0f1df (127.8.131.1:35805)
I20260809 06:30:17.544195 3832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1de250695b44403bae83d78181e0ebf" candidate_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db63456ddd4468f9c6087c31b90eaf2"
I20260809 06:30:17.544270 3647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1de250695b44403bae83d78181e0ebf" candidate_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be09ecfd302b442dbbf45943b0a0f1df"
I20260809 06:30:17.544390 3832 raft_consensus.cc:3037] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:17.544538 3647 raft_consensus.cc:3037] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:17.546653 3832 raft_consensus.cc:2454] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55fa44bd0b67461c8af9c5589e61b7e1 in term 1.
I20260809 06:30:17.546805 3647 raft_consensus.cc:2454] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55fa44bd0b67461c8af9c5589e61b7e1 in term 1.
I20260809 06:30:17.547078 3928 leader_election.cc:302] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55fa44bd0b67461c8af9c5589e61b7e1, 9db63456ddd4468f9c6087c31b90eaf2; no voters:
I20260809 06:30:17.547353 4111 raft_consensus.cc:2781] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:17.547863 4111 raft_consensus.cc:686] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 1 LEADER]: Becoming Leader. State: Replica: 55fa44bd0b67461c8af9c5589e61b7e1, State: Running, Role: LEADER
I20260809 06:30:17.548314 4111 consensus_queue.cc:237] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } }
I20260809 06:30:17.552925 3436 catalog_manager.cc:5495] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55fa44bd0b67461c8af9c5589e61b7e1 (127.8.131.3). New cstate: current_term: 1 leader_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55fa44bd0b67461c8af9c5589e61b7e1" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 37023 } health_report { overall_health: HEALTHY } } }
I20260809 06:30:17.572144 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.567319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.8.131.3:37023)
0809 06:30:17.567477 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.567481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.567498 (+ 17us) server_negotiation.cc:407] Connection header received
0809 06:30:17.567698 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.567710 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.567792 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.567913 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:30:17.568249 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.568965 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.570199 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.570567 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.570652 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.571094 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:30:17.571531 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:30:17.571714 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.571819 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.571854 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 06:30:17.578109 3711 tablet.cc:2369] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:30:17.580081 3647 raft_consensus.cc:1261] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [term 1 FOLLOWER]: Refusing update from remote peer 55fa44bd0b67461c8af9c5589e61b7e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:17.580094 3832 raft_consensus.cc:1261] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [term 1 FOLLOWER]: Refusing update from remote peer 55fa44bd0b67461c8af9c5589e61b7e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:17.581035 4114 consensus_queue.cc:1035] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be09ecfd302b442dbbf45943b0a0f1df" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:17.581406 4111 consensus_queue.cc:1035] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9db63456ddd4468f9c6087c31b90eaf2" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 39733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:17.591080 4117 mvcc.cc:205] Tried to move back new op lower bound from 7316508743998046208 to 7316508743881515008. Current Snapshot: MvccSnapshot[applied={T|T < 7316508743998046208}]
I20260809 06:30:17.599766 4118 mvcc.cc:205] Tried to move back new op lower bound from 7316508743998046208 to 7316508743881515008. Current Snapshot: MvccSnapshot[applied={T|T < 7316508743998046208}]
I20260809 06:30:17.612221 3434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:17.612561 3434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:17.616464 3434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f571446f1b634de289af27bf1fbd4901: Sending DeleteTablet for 3 replicas of tablet c1de250695b44403bae83d78181e0ebf
I20260809 06:30:17.617630 3995 tablet_service.cc:1514] Processing DeleteTablet for tablet c1de250695b44403bae83d78181e0ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:17 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:40662
I20260809 06:30:17.617723 3805 tablet_service.cc:1514] Processing DeleteTablet for tablet c1de250695b44403bae83d78181e0ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:17 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:36324
I20260809 06:30:17.618422 3616 tablet_service.cc:1514] Processing DeleteTablet for tablet c1de250695b44403bae83d78181e0ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:17 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:38196
I20260809 06:30:17.621112 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 3529
I20260809 06:30:17.627085 4133 tablet_replica.cc:330] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: stopping tablet replica
I20260809 06:30:17.627665 4133 raft_consensus.cc:2229] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:17.628334 4133 raft_consensus.cc:2258] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:17.630815 4133 ts_tablet_manager.cc:1905] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:17.632381 4131 tablet_replica.cc:330] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: stopping tablet replica
I20260809 06:30:17.632767 4131 raft_consensus.cc:2229] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:30:17.633448 4131 raft_consensus.cc:2258] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:17.634671 4132 tablet_replica.cc:330] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: stopping tablet replica
I20260809 06:30:17.635465 4132 raft_consensus.cc:2229] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:17.636045 4131 ts_tablet_manager.cc:1905] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:17.636221 4132 raft_consensus.cc:2258] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:17.641124 4133 ts_tablet_manager.cc:1918] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:17.641372 4133 log.cc:1196] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/c1de250695b44403bae83d78181e0ebf
I20260809 06:30:17.642025 4133 ts_tablet_manager.cc:1939] T c1de250695b44403bae83d78181e0ebf P be09ecfd302b442dbbf45943b0a0f1df: Deleting consensus metadata
I20260809 06:30:17.644019 3418 catalog_manager.cc:4841] TS be09ecfd302b442dbbf45943b0a0f1df (127.8.131.1:35805): tablet c1de250695b44403bae83d78181e0ebf (table test-table [id=67fb2f98e4c54f5eb647207bdbe659e8]) successfully deleted
I20260809 06:30:17.643748 4131 ts_tablet_manager.cc:1918] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:17.645197 4131 log.cc:1196] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/c1de250695b44403bae83d78181e0ebf
I20260809 06:30:17.645860 4131 ts_tablet_manager.cc:1939] T c1de250695b44403bae83d78181e0ebf P 55fa44bd0b67461c8af9c5589e61b7e1: Deleting consensus metadata
I20260809 06:30:17.646699 4132 ts_tablet_manager.cc:1905] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:17.647714 3415 catalog_manager.cc:4841] TS 55fa44bd0b67461c8af9c5589e61b7e1 (127.8.131.3:37023): tablet c1de250695b44403bae83d78181e0ebf (table test-table [id=67fb2f98e4c54f5eb647207bdbe659e8]) successfully deleted
Aug 09 06:30:17 dist-test-slave-mjjr krb5kdc[3377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:30:17.653653 4132 ts_tablet_manager.cc:1918] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:17.653825 4132 log.cc:1196] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/c1de250695b44403bae83d78181e0ebf
I20260809 06:30:17.654364 4132 ts_tablet_manager.cc:1939] T c1de250695b44403bae83d78181e0ebf P 9db63456ddd4468f9c6087c31b90eaf2: Deleting consensus metadata
I20260809 06:30:17.655437 4099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.622648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.8.131.1:35805)
0809 06:30:17.622862 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.622869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.626651 (+ 3782us) server_negotiation.cc:407] Connection header received
0809 06:30:17.627533 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.627540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.627683 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.627774 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:17.629897 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.631378 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.633454 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.635418 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.635526 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.648973 (+ 13447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:17.648984 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:17.648996 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:17.649036 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:17.652045 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.652838 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.652845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.652848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.652989 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.653547 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.653552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.653556 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.654018 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:17.654332 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.654723 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.654946 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 06:30:17.656178 3703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:17.656297 3418 catalog_manager.cc:4841] TS 9db63456ddd4468f9c6087c31b90eaf2 (127.8.131.2:39733): tablet c1de250695b44403bae83d78181e0ebf (table test-table [id=67fb2f98e4c54f5eb647207bdbe659e8]) successfully deleted
I20260809 06:30:17.748237 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 3529
I20260809 06:30:17.779026 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 3718
Aug 09 06:30:17 dist-test-slave-mjjr krb5kdc[3377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:30:17.796319 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.779903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36332 (local address 127.8.131.2:39733)
0809 06:30:17.780105 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.780111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.780137 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:30:17.780225 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.780231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.780324 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.780452 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:17.781763 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.782936 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.784492 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.785140 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.785296 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.790118 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:17.790134 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:17.790157 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:17.790209 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:17.793300 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.794209 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.794216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.794219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.794352 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.794791 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.794794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.794796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.795190 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:17.795428 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.795846 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.796007 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 06:30:17.796952 3885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:17.913331 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 3718
I20260809 06:30:17.937448 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 3903
Aug 09 06:30:17 dist-test-slave-mjjr krb5kdc[3377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:30:17.955055 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:17.938306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.8.131.3:37023)
0809 06:30:17.938488 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:17.938492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:17.938548 (+ 56us) server_negotiation.cc:407] Connection header received
0809 06:30:17.938799 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:17.938803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:17.938865 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:17.938950 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:17.940973 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.941916 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.943213 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:17.943961 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:17.944107 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:17.948817 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:17.948828 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:17.948842 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:17.948887 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:17.952042 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.952828 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.952838 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.952841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.953025 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:17.953426 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:17.953432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:17.953436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:17.953890 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:17.954244 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:17.954463 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 06:30:17.954682 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:30:17.955864 4071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:18.077194 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 3903
I20260809 06:30:18.104589 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 3394
I20260809 06:30:18.105741 3479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:18.217659 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 3394
2026-08-09T06:30:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4594 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T06:30:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:18Z Disabled control of system clock
I20260809 06:30:18.309994 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:43687
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:43687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 06:30:18.652493 4144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:18.652752 4144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:18.652900 4144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:18.660048 4144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:18.660158 4144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:18.660249 4144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:18.660356 4144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:18.660408 4144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:18.669361 4144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:43687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:43687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:18.670588 4144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:18.672403 4144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:18.684536 4154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:18.684443 4156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:18.684510 4159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:18.685434 4144 server_base.cc:1034] running on GCE node
I20260809 06:30:18.686208 4144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:18.687610 4144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:18.688869 4144 hybrid_clock.cc:648] HybridClock initialized: now 1786257018688817 us; error 52 us; skew 500 ppm
I20260809 06:30:18.689239 4144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:18.691438 4144 webserver.cc:460] Webserver started at http://127.8.131.62:35455/ using document root <none> and password file <none>
I20260809 06:30:18.692157 4144 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:18.692272 4144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:18.692603 4144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:18.695487 4144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "d1b8dcb3e62c491c961949e03236b6e9"
format_stamp: "Formatted at 2026-08-09 06:30:18 on dist-test-slave-mjjr"
server_key: "29cd9c7d5943daddd849118967b4c42d"
server_key_iv: "7a6bdcccddb3161c0d8adfd198198ba9"
server_key_version: "encryptionkey@0"
I20260809 06:30:18.696398 4144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "d1b8dcb3e62c491c961949e03236b6e9"
format_stamp: "Formatted at 2026-08-09 06:30:18 on dist-test-slave-mjjr"
server_key: "29cd9c7d5943daddd849118967b4c42d"
server_key_iv: "7a6bdcccddb3161c0d8adfd198198ba9"
server_key_version: "encryptionkey@0"
I20260809 06:30:18.701552 4144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:30:18.705241 4170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:18.706614 4144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:30:18.706817 4144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "d1b8dcb3e62c491c961949e03236b6e9"
format_stamp: "Formatted at 2026-08-09 06:30:18 on dist-test-slave-mjjr"
server_key: "29cd9c7d5943daddd849118967b4c42d"
server_key_iv: "7a6bdcccddb3161c0d8adfd198198ba9"
server_key_version: "encryptionkey@0"
I20260809 06:30:18.707041 4144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:18.731606 4144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:18.732542 4144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:18.732872 4144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:18.751941 4144 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:43687
I20260809 06:30:18.751961 4233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:43687 every 8 connection(s)
I20260809 06:30:18.753342 4144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:18.755194 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 4144
I20260809 06:30:18.755388 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:18.755896 8716 external_mini_cluster.cc:1428] Setting key 03e7b6577369f0f7f2633ba34d9eee07
I20260809 06:30:18.757702 4234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:18.771299 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:18.757862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42088 (local address 127.8.131.62:43687)
0809 06:30:18.758934 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:18.758950 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:18.759005 (+ 55us) server_negotiation.cc:407] Connection header received
0809 06:30:18.761044 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:18.761070 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:18.761673 (+ 603us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:18.762106 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:18.762644 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:18.764767 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:18.766084 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:18.767159 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:18.767342 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:18.768515 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:18.768525 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:18.768547 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:18.769288 (+ 741us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:18.769387 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:18.769417 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:18.769557 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:18.769724 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 06:30:18.770055 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":320,"threads_started":1}
I20260809 06:30:18.772569 4234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9: Bootstrap starting.
I20260809 06:30:18.775563 4234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:18.776582 4234 log.cc:824] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:18.779163 4234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9: No bootstrap required, opened a new log
I20260809 06:30:18.786212 4234 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b8dcb3e62c491c961949e03236b6e9" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 43687 } }
I20260809 06:30:18.786540 4234 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:18.786655 4234 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b8dcb3e62c491c961949e03236b6e9, State: Initialized, Role: FOLLOWER
I20260809 06:30:18.787274 4234 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b8dcb3e62c491c961949e03236b6e9" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 43687 } }
I20260809 06:30:18.787514 4234 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:18.787689 4234 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:18.787878 4234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:18.789944 4234 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b8dcb3e62c491c961949e03236b6e9" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 43687 } }
I20260809 06:30:18.790493 4234 leader_election.cc:302] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1b8dcb3e62c491c961949e03236b6e9; no voters:
I20260809 06:30:18.791003 4234 leader_election.cc:288] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:18.791193 4241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:18.792178 4241 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [term 1 LEADER]: Becoming Leader. State: Replica: d1b8dcb3e62c491c961949e03236b6e9, State: Running, Role: LEADER
I20260809 06:30:18.792847 4241 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b8dcb3e62c491c961949e03236b6e9" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 43687 } }
I20260809 06:30:18.793150 4234 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:18.795379 4245 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1b8dcb3e62c491c961949e03236b6e9. Latest consensus state: current_term: 1 leader_uuid: "d1b8dcb3e62c491c961949e03236b6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b8dcb3e62c491c961949e03236b6e9" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 43687 } } }
I20260809 06:30:18.795568 4244 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1b8dcb3e62c491c961949e03236b6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b8dcb3e62c491c961949e03236b6e9" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 43687 } } }
I20260809 06:30:18.795869 4245 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:18.795872 4244 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:18.796958 4248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:18.801520 4248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:18.810988 4248 catalog_manager.cc:1347] Generated new cluster ID: ada20b84ea674f48b824e2d491ef498f
I20260809 06:30:18.811117 4248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:18.838912 4248 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:18.840210 4248 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:18.852425 4248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9: Generated new TSK 0
I20260809 06:30:18.853223 4248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:18.858636 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:43687
--builtin_ntp_servers=127.8.131.20:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 06:30:19.198210 4269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:19.198478 4269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:19.198565 4269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:19.205665 4269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:19.205757 4269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:19.205863 4269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:30:19.214818 4269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:43687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:19.216030 4269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:19.217881 4269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:19.230695 4279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:19.230654 4277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:19.231127 4281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:19.231689 4269 server_base.cc:1034] running on GCE node
I20260809 06:30:19.232268 4269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:19.233107 4269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:19.234360 4269 hybrid_clock.cc:648] HybridClock initialized: now 1786257019234310 us; error 49 us; skew 500 ppm
I20260809 06:30:19.234694 4269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:19.236943 4269 webserver.cc:460] Webserver started at http://127.8.131.1:45165/ using document root <none> and password file <none>
I20260809 06:30:19.237645 4269 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:19.237782 4269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:19.238119 4269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:19.240923 4269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "094fe3aa40f84e23b24735de0a09d710"
format_stamp: "Formatted at 2026-08-09 06:30:19 on dist-test-slave-mjjr"
server_key: "bef8bae6386b5d13350d22483b5a6bd0"
server_key_iv: "bfdb1c89f84f23094ababf14c38ec604"
server_key_version: "encryptionkey@0"
I20260809 06:30:19.241719 4269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "094fe3aa40f84e23b24735de0a09d710"
format_stamp: "Formatted at 2026-08-09 06:30:19 on dist-test-slave-mjjr"
server_key: "bef8bae6386b5d13350d22483b5a6bd0"
server_key_iv: "bfdb1c89f84f23094ababf14c38ec604"
server_key_version: "encryptionkey@0"
I20260809 06:30:19.246750 4269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 06:30:19.250097 4287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:19.251345 4269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:30:19.251528 4269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "094fe3aa40f84e23b24735de0a09d710"
format_stamp: "Formatted at 2026-08-09 06:30:19 on dist-test-slave-mjjr"
server_key: "bef8bae6386b5d13350d22483b5a6bd0"
server_key_iv: "bfdb1c89f84f23094ababf14c38ec604"
server_key_version: "encryptionkey@0"
I20260809 06:30:19.251794 4269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:19.262729 4269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:19.263461 4269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:19.263885 4269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:19.264900 4269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:19.266304 4269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:19.266435 4269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:19.266579 4269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:19.266690 4269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:19.300262 4269 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:38281
I20260809 06:30:19.300321 4468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:38281 every 8 connection(s)
I20260809 06:30:19.301542 4269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:30:19.309191 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 4269
I20260809 06:30:19.309392 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:30:19.309926 8716 external_mini_cluster.cc:1428] Setting key 94d290cc124177391f270862117041fa
I20260809 06:30:19.313225 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:43687
--builtin_ntp_servers=127.8.131.20:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 06:30:19.314742 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:19.304101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:50471 (local address 127.8.131.62:43687)
0809 06:30:19.304551 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:19.304555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:19.305711 (+ 1156us) server_negotiation.cc:407] Connection header received
0809 06:30:19.307503 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:19.307509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:19.307597 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:19.307747 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:19.308778 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:19.309592 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:19.311617 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:19.312235 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:19.312337 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:19.314008 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:19.314014 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:19.314017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:19.314034 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:19.314072 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:19.314084 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:19.314158 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:19.314416 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 06:30:19.314505 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":241,"threads_started":1}
I20260809 06:30:19.316195 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:19.304682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:43687 (local address 127.8.131.1:50471)
0809 06:30:19.305543 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:19.305582 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:19.307225 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:19.307969 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:19.307977 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:19.308062 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:19.308599 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:19.308615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:19.309770 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:19.309774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:19.311455 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:19.311469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:19.312467 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:19.312475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:19.312751 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:19.313644 (+ 893us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:30:19.313650 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:19.313706 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:30:19.313801 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:19.314189 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:19.314202 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:19.314214 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 06:30:19.314415 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 06:30:19.314831 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":280,"threads_started":1}
I20260809 06:30:19.317741 4469 heartbeater.cc:347] Connected to a master server at 127.8.131.62:43687
I20260809 06:30:19.318032 4469 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:19.318737 4469 heartbeater.cc:511] Master 127.8.131.62:43687 requested a full tablet report, sending...
I20260809 06:30:19.320979 4192 ts_manager.cc:194] Registered new tserver with Master: 094fe3aa40f84e23b24735de0a09d710 (127.8.131.1:38281)
I20260809 06:30:19.323173 4192 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.1:50471
W20260809 06:30:19.653337 4476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:19.653616 4476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:19.653801 4476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:19.661113 4476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:19.661208 4476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:19.661377 4476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:30:19.670352 4476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:43687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:19.671563 4476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:19.673444 4476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:19.687055 4483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:19.687103 4484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:19.687129 4486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:19.687849 4476 server_base.cc:1034] running on GCE node
I20260809 06:30:19.688465 4476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:19.689368 4476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:19.690572 4476 hybrid_clock.cc:648] HybridClock initialized: now 1786257019690525 us; error 41 us; skew 500 ppm
I20260809 06:30:19.690877 4476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:19.693197 4476 webserver.cc:460] Webserver started at http://127.8.131.2:45167/ using document root <none> and password file <none>
I20260809 06:30:19.693833 4476 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:19.693949 4476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:19.694285 4476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:19.697000 4476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "a01d282a59524146a8b1a2641b3bfa9e"
format_stamp: "Formatted at 2026-08-09 06:30:19 on dist-test-slave-mjjr"
server_key: "fcb6b2d7a22f807876f0a6fbbe6676fa"
server_key_iv: "2029f0e27be7bdd13507f3f0f6ae3658"
server_key_version: "encryptionkey@0"
I20260809 06:30:19.697796 4476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "a01d282a59524146a8b1a2641b3bfa9e"
format_stamp: "Formatted at 2026-08-09 06:30:19 on dist-test-slave-mjjr"
server_key: "fcb6b2d7a22f807876f0a6fbbe6676fa"
server_key_iv: "2029f0e27be7bdd13507f3f0f6ae3658"
server_key_version: "encryptionkey@0"
I20260809 06:30:19.702555 4476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 06:30:19.705967 4497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:19.707212 4476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:30:19.707403 4476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "a01d282a59524146a8b1a2641b3bfa9e"
format_stamp: "Formatted at 2026-08-09 06:30:19 on dist-test-slave-mjjr"
server_key: "fcb6b2d7a22f807876f0a6fbbe6676fa"
server_key_iv: "2029f0e27be7bdd13507f3f0f6ae3658"
server_key_version: "encryptionkey@0"
I20260809 06:30:19.707682 4476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:19.721414 4476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:19.722162 4476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:19.722445 4476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:19.723395 4476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:19.724864 4476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:19.724953 4476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:19.725123 4476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:19.725251 4476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:19.760205 4476 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:34641
I20260809 06:30:19.760246 4659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:34641 every 8 connection(s)
I20260809 06:30:19.761508 4476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:30:19.764635 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 4476
I20260809 06:30:19.764833 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:30:19.765367 8716 external_mini_cluster.cc:1428] Setting key d69c98fd8805aa525cda8cd1944c5cd0
I20260809 06:30:19.768751 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:43687
--builtin_ntp_servers=127.8.131.20:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 06:30:19.775403 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:19.764227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:35365 (local address 127.8.131.62:43687)
0809 06:30:19.766065 (+ 1838us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:19.766071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:19.766234 (+ 163us) server_negotiation.cc:407] Connection header received
0809 06:30:19.768724 (+ 2490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:19.768730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:19.768815 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:19.768909 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:19.770281 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:19.770999 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:19.772617 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:19.773034 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:19.773176 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:19.774549 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:19.774554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:19.774557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:19.774575 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:19.774605 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:19.774617 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:19.774712 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:19.774974 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 06:30:19.775077 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1712}
I20260809 06:30:19.776417 4662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:19.764880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:43687 (local address 127.8.131.2:35365)
0809 06:30:19.766057 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:19.766110 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:19.768419 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:19.769277 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:19.769288 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:19.769398 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:19.770108 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:19.770126 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:19.771163 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:19.771170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:19.772469 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:19.772482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:19.773263 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:19.773271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:19.773503 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:19.774233 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:30:19.774236 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:19.774275 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:30:19.774352 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:19.774713 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:19.774726 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:19.774736 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 06:30:19.774947 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 06:30:19.775326 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":340,"threads_started":1}
I20260809 06:30:19.777854 4660 heartbeater.cc:347] Connected to a master server at 127.8.131.62:43687
I20260809 06:30:19.778203 4660 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:19.779161 4660 heartbeater.cc:511] Master 127.8.131.62:43687 requested a full tablet report, sending...
I20260809 06:30:19.780788 4192 ts_manager.cc:194] Registered new tserver with Master: a01d282a59524146a8b1a2641b3bfa9e (127.8.131.2:34641)
I20260809 06:30:19.781620 4192 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.2:35365
W20260809 06:30:20.120731 4663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:20.121028 4663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:20.121213 4663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:20.128679 4663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:20.128851 4663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:20.129050 4663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:30:20.138439 4663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:43687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:20.139923 4663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:20.141836 4663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:20.155759 4670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:20.155793 4673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:20.155725 4669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:20.156625 4663 server_base.cc:1034] running on GCE node
I20260809 06:30:20.157280 4663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:20.158164 4663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:20.159361 4663 hybrid_clock.cc:648] HybridClock initialized: now 1786257020159313 us; error 57 us; skew 500 ppm
I20260809 06:30:20.159621 4663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:20.161891 4663 webserver.cc:460] Webserver started at http://127.8.131.3:40471/ using document root <none> and password file <none>
I20260809 06:30:20.162536 4663 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:20.162626 4663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:20.162892 4663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:20.165629 4663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "f85cfe2570174800ae6c215547eade5f"
format_stamp: "Formatted at 2026-08-09 06:30:20 on dist-test-slave-mjjr"
server_key: "e8c7b541c4513c8ac6f5485dbb0de5c8"
server_key_iv: "a850b5a0b82c9862074909541edb3111"
server_key_version: "encryptionkey@0"
I20260809 06:30:20.166405 4663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "f85cfe2570174800ae6c215547eade5f"
format_stamp: "Formatted at 2026-08-09 06:30:20 on dist-test-slave-mjjr"
server_key: "e8c7b541c4513c8ac6f5485dbb0de5c8"
server_key_iv: "a850b5a0b82c9862074909541edb3111"
server_key_version: "encryptionkey@0"
I20260809 06:30:20.171516 4663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 06:30:20.175073 4679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:20.176391 4663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:20.176600 4663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "f85cfe2570174800ae6c215547eade5f"
format_stamp: "Formatted at 2026-08-09 06:30:20 on dist-test-slave-mjjr"
server_key: "e8c7b541c4513c8ac6f5485dbb0de5c8"
server_key_iv: "a850b5a0b82c9862074909541edb3111"
server_key_version: "encryptionkey@0"
I20260809 06:30:20.176823 4663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:20.194216 4663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:20.195009 4663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:20.195305 4663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:20.196372 4663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:20.197826 4663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:20.197969 4663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:20.198103 4663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:20.198195 4663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:20.232779 4663 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:34855
I20260809 06:30:20.232870 4849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:34855 every 8 connection(s)
I20260809 06:30:20.234162 4663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:30:20.241641 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 4663
I20260809 06:30:20.241848 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:30:20.242305 8716 external_mini_cluster.cc:1428] Setting key c2ed9f6bee7b16a0ecdf62779127cfe2
I20260809 06:30:20.246881 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.236566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:41785 (local address 127.8.131.62:43687)
0809 06:30:20.236759 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.236763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.238088 (+ 1325us) server_negotiation.cc:407] Connection header received
0809 06:30:20.239905 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.239914 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.240005 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.240135 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:20.241186 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.242063 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.243989 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.244602 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.244703 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.246197 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:20.246202 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:20.246205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:20.246221 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:20.246253 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:20.246265 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:20.246341 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.246553 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.246656 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 06:30:20.247949 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.237031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:43687 (local address 127.8.131.3:41785)
0809 06:30:20.237916 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.237965 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:20.239569 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:20.240363 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:20.240371 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:20.240454 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:20.241005 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.241021 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.242233 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.242238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.243815 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.243828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.244831 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.244838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.245145 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.245910 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:30:20.245913 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:20.245950 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:30:20.246029 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:20.246361 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:20.246368 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:20.246374 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 06:30:20.246526 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 06:30:20.246799 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":283,"threads_started":1}
I20260809 06:30:20.249238 4851 heartbeater.cc:347] Connected to a master server at 127.8.131.62:43687
I20260809 06:30:20.249539 4851 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:20.250232 4851 heartbeater.cc:511] Master 127.8.131.62:43687 requested a full tablet report, sending...
I20260809 06:30:20.251552 4192 ts_manager.cc:194] Registered new tserver with Master: f85cfe2570174800ae6c215547eade5f (127.8.131.3:34855)
I20260809 06:30:20.252398 4192 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.3:41785
I20260809 06:30:20.257455 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:30:20.267622 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.261469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42092 (local address 127.8.131.62:43687)
0809 06:30:20.261622 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.261626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.262149 (+ 523us) server_negotiation.cc:407] Connection header received
0809 06:30:20.262341 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.262346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.262414 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.262518 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:20.263041 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.263836 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.265142 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.265586 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.265732 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.266885 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:20.266890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:20.266893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:20.266909 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:20.266937 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:20.266948 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:20.267094 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.267288 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.267355 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:30:20.271492 4192 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:30:20.278159 4192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:30:20.312618 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.290434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.8.131.1:38281)
0809 06:30:20.303310 (+ 12876us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.303318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.303348 (+ 30us) server_negotiation.cc:407] Connection header received
0809 06:30:20.303445 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.303455 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.303745 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.303928 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:20.305380 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.306989 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.309691 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.311245 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.311434 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.311764 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.312067 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.312239 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12671,"thread_start_us":13853,"threads_started":1}
I20260809 06:30:20.312696 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.290131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38281 (local address 127.0.0.1:34228)
0809 06:30:20.290867 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.290919 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:20.291096 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:20.304228 (+ 13132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:20.304235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:20.304284 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:20.305191 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.305203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.307193 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.307198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.309440 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.309456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.311526 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.311532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.311768 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.311837 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 06:30:20.312031 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 06:30:20.312283 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":467,"spinlock_wait_cycles":8320,"thread_start_us":1500,"threads_started":1}
I20260809 06:30:20.312919 4874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.293094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46016 (local address 127.8.131.2:34641)
0809 06:30:20.301622 (+ 8528us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.301635 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.303214 (+ 1579us) server_negotiation.cc:407] Connection header received
0809 06:30:20.303535 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.303542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.303743 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.303916 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:20.305016 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.306750 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.309897 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.311265 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.311386 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.311691 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.312353 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.312548 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8279,"thread_start_us":15214,"threads_started":1}
I20260809 06:30:20.313907 4873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.292546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:34641 (local address 127.0.0.1:46016)
0809 06:30:20.303081 (+ 10535us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.303102 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:20.303336 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:20.304228 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:20.304235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:20.304284 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:20.304767 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.304779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.306929 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.306936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.309703 (+ 2767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.309716 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.311880 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.311887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.312114 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.312157 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 06:30:20.313302 (+ 1145us) client_negotiation.cc:240] Negotiation successful
0809 06:30:20.313435 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10431,"mutex_wait_us":4,"spinlock_wait_cycles":6144,"thread_start_us":15650,"threads_started":1}
I20260809 06:30:20.318470 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.290746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:34855 (local address 127.0.0.1:53834)
0809 06:30:20.304513 (+ 13767us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.304530 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:20.304651 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:20.311047 (+ 6396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:20.311052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:20.311079 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:20.311616 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.311629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.313790 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.313797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.316383 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.316398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.317695 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.317702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.317872 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.317913 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 06:30:20.318037 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 06:30:20.318164 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11918,"thread_start_us":11762,"threads_started":1}
I20260809 06:30:20.318646 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.291180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53834 (local address 127.8.131.3:34855)
0809 06:30:20.310500 (+ 19320us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.310510 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.310543 (+ 33us) server_negotiation.cc:407] Connection header received
0809 06:30:20.310617 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.310622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.310868 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.311067 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:20.311906 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.313610 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.316577 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.317483 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.317658 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.317937 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.318090 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.318251 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19145,"thread_start_us":18972,"threads_started":1}
I20260809 06:30:20.318244 4563 tablet_service.cc:1467] Processing CreateTablet for tablet dcbb86bd2a634629b884f2ead85272e5 (DEFAULT_TABLE table=test-table [id=b6e3b8a03e2d4e3d99e0f22b7ab3e131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:20.318244 4362 tablet_service.cc:1467] Processing CreateTablet for tablet dcbb86bd2a634629b884f2ead85272e5 (DEFAULT_TABLE table=test-table [id=b6e3b8a03e2d4e3d99e0f22b7ab3e131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:20.319772 4362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbb86bd2a634629b884f2ead85272e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:20.319793 4563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbb86bd2a634629b884f2ead85272e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:20.322263 4750 tablet_service.cc:1467] Processing CreateTablet for tablet dcbb86bd2a634629b884f2ead85272e5 (DEFAULT_TABLE table=test-table [id=b6e3b8a03e2d4e3d99e0f22b7ab3e131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:20.323424 4750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbb86bd2a634629b884f2ead85272e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:20.326534 4469 heartbeater.cc:503] Master 127.8.131.62:43687 was elected leader, sending a full tablet report...
I20260809 06:30:20.334651 4875 tablet_bootstrap.cc:492] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: Bootstrap starting.
I20260809 06:30:20.335721 4878 tablet_bootstrap.cc:492] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: Bootstrap starting.
I20260809 06:30:20.336747 4879 tablet_bootstrap.cc:492] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: Bootstrap starting.
I20260809 06:30:20.339533 4879 tablet_bootstrap.cc:654] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:20.339720 4878 tablet_bootstrap.cc:654] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:20.339888 4875 tablet_bootstrap.cc:654] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:20.340865 4879 log.cc:824] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:20.341014 4875 log.cc:824] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:20.341008 4878 log.cc:824] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:20.344501 4879 tablet_bootstrap.cc:492] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: No bootstrap required, opened a new log
I20260809 06:30:20.344734 4878 tablet_bootstrap.cc:492] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: No bootstrap required, opened a new log
I20260809 06:30:20.344776 4879 ts_tablet_manager.cc:1397] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 06:30:20.345013 4878 ts_tablet_manager.cc:1397] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 06:30:20.345096 4875 tablet_bootstrap.cc:492] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: No bootstrap required, opened a new log
I20260809 06:30:20.345409 4875 ts_tablet_manager.cc:1397] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260809 06:30:20.352298 4879 raft_consensus.cc:348] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } }
I20260809 06:30:20.352654 4879 raft_consensus.cc:374] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:20.352783 4879 raft_consensus.cc:729] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a01d282a59524146a8b1a2641b3bfa9e, State: Initialized, Role: FOLLOWER
I20260809 06:30:20.353369 4879 consensus_queue.cc:260] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } }
I20260809 06:30:20.354470 4660 heartbeater.cc:503] Master 127.8.131.62:43687 was elected leader, sending a full tablet report...
I20260809 06:30:20.355186 4879 ts_tablet_manager.cc:1428] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 06:30:20.355957 4875 raft_consensus.cc:348] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } }
I20260809 06:30:20.356369 4875 raft_consensus.cc:374] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:20.356469 4875 raft_consensus.cc:729] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f85cfe2570174800ae6c215547eade5f, State: Initialized, Role: FOLLOWER
I20260809 06:30:20.356976 4878 raft_consensus.cc:348] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } }
I20260809 06:30:20.357117 4875 consensus_queue.cc:260] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } }
I20260809 06:30:20.357388 4878 raft_consensus.cc:374] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:20.357493 4878 raft_consensus.cc:729] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094fe3aa40f84e23b24735de0a09d710, State: Initialized, Role: FOLLOWER
I20260809 06:30:20.358151 4878 consensus_queue.cc:260] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } }
I20260809 06:30:20.360260 4878 ts_tablet_manager.cc:1428] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260809 06:30:20.364393 4851 heartbeater.cc:503] Master 127.8.131.62:43687 was elected leader, sending a full tablet report...
I20260809 06:30:20.364975 4875 ts_tablet_manager.cc:1428] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
I20260809 06:30:20.467006 4889 raft_consensus.cc:482] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:30:20.467326 4889 raft_consensus.cc:504] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } }
I20260809 06:30:20.469095 4889 leader_election.cc:288] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a01d282a59524146a8b1a2641b3bfa9e (127.8.131.2:34641), f85cfe2570174800ae6c215547eade5f (127.8.131.3:34855)
I20260809 06:30:20.476135 4874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.469564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:59353 (local address 127.8.131.2:34641)
0809 06:30:20.469775 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.469783 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.470533 (+ 750us) server_negotiation.cc:407] Connection header received
0809 06:30:20.470738 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.470746 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.470846 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.470917 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:20.471575 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.472331 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.474402 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.475167 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.475291 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.475389 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.475711 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.475852 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:30:20.476138 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.469683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:51207 (local address 127.8.131.3:34855)
0809 06:30:20.469900 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.469909 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.470626 (+ 717us) server_negotiation.cc:407] Connection header received
0809 06:30:20.470840 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.470844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.470935 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.471030 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:20.471734 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.472479 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.474441 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.475192 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.475357 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.475451 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.475756 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.475884 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 06:30:20.476289 4891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.469604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:34855 (local address 127.8.131.1:51207)
0809 06:30:20.470523 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.470543 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:20.470683 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:20.471081 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:20.471085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:20.471117 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:20.471522 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.471536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.472650 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.472654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.474189 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.474201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.475416 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.475425 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.475556 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.475582 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 06:30:20.475712 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 06:30:20.475897 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":115,"thread_start_us":531,"threads_started":1}
I20260809 06:30:20.476337 4890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.469567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:34641 (local address 127.8.131.1:59353)
0809 06:30:20.470401 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.470422 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:20.470548 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:20.470972 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:20.470980 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:20.471012 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:20.471412 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.471422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.472472 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.472477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.474189 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:20.474201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.475369 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.475375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:20.475543 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.475567 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 06:30:20.475709 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 06:30:20.475890 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":87,"thread_start_us":468,"threads_started":1}
I20260809 06:30:20.477515 4781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbb86bd2a634629b884f2ead85272e5" candidate_uuid: "094fe3aa40f84e23b24735de0a09d710" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85cfe2570174800ae6c215547eade5f" is_pre_election: true
I20260809 06:30:20.477711 4593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbb86bd2a634629b884f2ead85272e5" candidate_uuid: "094fe3aa40f84e23b24735de0a09d710" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01d282a59524146a8b1a2641b3bfa9e" is_pre_election: true
I20260809 06:30:20.478174 4781 raft_consensus.cc:2454] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 094fe3aa40f84e23b24735de0a09d710 in term 0.
I20260809 06:30:20.478245 4593 raft_consensus.cc:2454] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 094fe3aa40f84e23b24735de0a09d710 in term 0.
I20260809 06:30:20.478957 4288 leader_election.cc:302] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094fe3aa40f84e23b24735de0a09d710, a01d282a59524146a8b1a2641b3bfa9e; no voters:
I20260809 06:30:20.479415 4889 raft_consensus.cc:2781] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:30:20.479581 4889 raft_consensus.cc:482] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:30:20.479722 4889 raft_consensus.cc:3037] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:20.482254 4889 raft_consensus.cc:504] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } }
I20260809 06:30:20.482841 4889 leader_election.cc:288] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [CANDIDATE]: Term 1 election: Requested vote from peers a01d282a59524146a8b1a2641b3bfa9e (127.8.131.2:34641), f85cfe2570174800ae6c215547eade5f (127.8.131.3:34855)
I20260809 06:30:20.483300 4781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbb86bd2a634629b884f2ead85272e5" candidate_uuid: "094fe3aa40f84e23b24735de0a09d710" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85cfe2570174800ae6c215547eade5f"
I20260809 06:30:20.483526 4781 raft_consensus.cc:3037] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:20.483594 4593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbb86bd2a634629b884f2ead85272e5" candidate_uuid: "094fe3aa40f84e23b24735de0a09d710" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01d282a59524146a8b1a2641b3bfa9e"
I20260809 06:30:20.483819 4593 raft_consensus.cc:3037] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [term 0 FOLLOWER]: Advancing to term 1
W20260809 06:30:20.485786 4852 tablet.cc:2369] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:30:20.486982 4781 raft_consensus.cc:2454] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 094fe3aa40f84e23b24735de0a09d710 in term 1.
I20260809 06:30:20.487109 4593 raft_consensus.cc:2454] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 094fe3aa40f84e23b24735de0a09d710 in term 1.
I20260809 06:30:20.487428 4290 leader_election.cc:302] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094fe3aa40f84e23b24735de0a09d710, f85cfe2570174800ae6c215547eade5f; no voters:
I20260809 06:30:20.487744 4889 raft_consensus.cc:2781] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:20.488138 4889 raft_consensus.cc:686] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 1 LEADER]: Becoming Leader. State: Replica: 094fe3aa40f84e23b24735de0a09d710, State: Running, Role: LEADER
I20260809 06:30:20.488603 4889 consensus_queue.cc:237] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } }
I20260809 06:30:20.493197 4189 catalog_manager.cc:5495] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 reported cstate change: term changed from 0 to 1, leader changed from <none> to 094fe3aa40f84e23b24735de0a09d710 (127.8.131.1). New cstate: current_term: 1 leader_uuid: "094fe3aa40f84e23b24735de0a09d710" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094fe3aa40f84e23b24735de0a09d710" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 } health_report { overall_health: UNKNOWN } } }
W20260809 06:30:20.513602 4661 tablet.cc:2369] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:30:20.540287 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.534560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34234 (local address 127.8.131.1:38281)
0809 06:30:20.534760 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.534768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.534853 (+ 85us) server_negotiation.cc:407] Connection header received
0809 06:30:20.535101 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.535105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.535198 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.535356 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:30:20.535825 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.536536 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.537996 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.538423 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.538571 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.539165 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:30:20.539664 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:30:20.539874 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.540021 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.540067 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:30:20.548382 4781 raft_consensus.cc:1261] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [term 1 FOLLOWER]: Refusing update from remote peer 094fe3aa40f84e23b24735de0a09d710: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:20.548426 4593 raft_consensus.cc:1261] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [term 1 FOLLOWER]: Refusing update from remote peer 094fe3aa40f84e23b24735de0a09d710: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:20.549360 4892 consensus_queue.cc:1035] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a01d282a59524146a8b1a2641b3bfa9e" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:20.549741 4889 consensus_queue.cc:1035] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f85cfe2570174800ae6c215547eade5f" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 06:30:20.554078 4470 tablet.cc:2369] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:30:20.575124 4192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:20.575440 4192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:20.579118 4192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1b8dcb3e62c491c961949e03236b6e9: Sending DeleteTablet for 3 replicas of tablet dcbb86bd2a634629b884f2ead85272e5
I20260809 06:30:20.580446 4362 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbb86bd2a634629b884f2ead85272e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:20 UTC) from {username='slave'} at 127.0.0.1:34228
I20260809 06:30:20.580574 4750 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbb86bd2a634629b884f2ead85272e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:20 UTC) from {username='slave'} at 127.0.0.1:53834
I20260809 06:30:20.581161 4563 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbb86bd2a634629b884f2ead85272e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:20 UTC) from {username='slave'} at 127.0.0.1:46016
I20260809 06:30:20.581892 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 4269
I20260809 06:30:20.590279 4917 tablet_replica.cc:330] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: stopping tablet replica
I20260809 06:30:20.590866 4918 tablet_replica.cc:330] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: stopping tablet replica
I20260809 06:30:20.590857 4917 raft_consensus.cc:2229] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:20.591434 4917 raft_consensus.cc:2258] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:20.591534 4918 raft_consensus.cc:2229] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:20.592370 4918 raft_consensus.cc:2258] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:20.593007 4913 tablet_replica.cc:330] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: stopping tablet replica
I20260809 06:30:20.593490 4913 raft_consensus.cc:2229] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:30:20.594316 4913 raft_consensus.cc:2258] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:20.595589 4917 ts_tablet_manager.cc:1905] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:20.596176 4918 ts_tablet_manager.cc:1905] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:20.600431 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.583041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.8.131.1:38281)
0809 06:30:20.583738 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.583746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.589942 (+ 6196us) server_negotiation.cc:407] Connection header received
0809 06:30:20.590355 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.590361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.590486 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.590600 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:20.592423 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.593708 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.595985 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.596613 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.596722 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.598411 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:20.598422 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:20.598438 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:20.599202 (+ 764us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:20.599257 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:20.599286 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:20.599375 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.599943 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.600055 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545}
I20260809 06:30:20.601336 4461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:20.603440 4918 ts_tablet_manager.cc:1918] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:20.603461 4917 ts_tablet_manager.cc:1918] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:20.603627 4917 log.cc:1196] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/dcbb86bd2a634629b884f2ead85272e5
I20260809 06:30:20.603684 4918 log.cc:1196] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/dcbb86bd2a634629b884f2ead85272e5
I20260809 06:30:20.604355 4917 ts_tablet_manager.cc:1939] T dcbb86bd2a634629b884f2ead85272e5 P f85cfe2570174800ae6c215547eade5f: Deleting consensus metadata
I20260809 06:30:20.604379 4918 ts_tablet_manager.cc:1939] T dcbb86bd2a634629b884f2ead85272e5 P a01d282a59524146a8b1a2641b3bfa9e: Deleting consensus metadata
I20260809 06:30:20.606455 4172 catalog_manager.cc:4841] TS f85cfe2570174800ae6c215547eade5f (127.8.131.3:34855): tablet dcbb86bd2a634629b884f2ead85272e5 (table test-table [id=b6e3b8a03e2d4e3d99e0f22b7ab3e131]) successfully deleted
I20260809 06:30:20.606876 4171 catalog_manager.cc:4841] TS a01d282a59524146a8b1a2641b3bfa9e (127.8.131.2:34641): tablet dcbb86bd2a634629b884f2ead85272e5 (table test-table [id=b6e3b8a03e2d4e3d99e0f22b7ab3e131]) successfully deleted
I20260809 06:30:20.701200 4913 ts_tablet_manager.cc:1905] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:20.706940 4913 ts_tablet_manager.cc:1918] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:20.707165 4913 log.cc:1196] T dcbb86bd2a634629b884f2ead85272e5 P 094fe3aa40f84e23b24735de0a09d710: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/dcbb86bd2a634629b884f2ead85272e5
I20260809 06:30:20.707669 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 4269
I20260809 06:30:20.732300 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 4476
W20260809 06:30:20.732391 4171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.131.1:38281 (error 108)
I20260809 06:30:20.734222 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.733777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38281 (local address 127.0.0.1:34238)
0809 06:30:20.733980 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.734092 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 06:30:20.734503 4171 catalog_manager.cc:4584] TS 094fe3aa40f84e23b24735de0a09d710 (127.8.131.1:38281): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dcbb86bd2a634629b884f2ead85272e5: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38281: connect: Connection refused (error 111)
I20260809 06:30:20.743276 4874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.733085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46024 (local address 127.8.131.2:34641)
0809 06:30:20.733281 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.733288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.733418 (+ 130us) server_negotiation.cc:407] Connection header received
0809 06:30:20.733623 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.733628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.733744 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.733863 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:20.734284 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.735156 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.736527 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.736985 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.737127 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.741873 (+ 4746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:20.741882 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:20.741895 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:20.742484 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:20.742533 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:20.742552 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:20.742691 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.742817 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.742962 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:30:20.744002 4652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:20.788051 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.787550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38281 (local address 127.0.0.1:34250)
0809 06:30:20.787796 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.787898 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 06:30:20.789146 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.788779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38281 (local address 127.0.0.1:34266)
0809 06:30:20.788959 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.789033 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 06:30:20.834372 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 4476
I20260809 06:30:20.849931 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.849379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38281 (local address 127.0.0.1:34282)
0809 06:30:20.849679 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.849771 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260809 06:30:20.851052 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.850719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38281 (local address 127.0.0.1:34292)
0809 06:30:20.850852 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.850920 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 06:30:20.856889 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 4663
I20260809 06:30:20.864030 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.857659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.8.131.3:34855)
0809 06:30:20.857851 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:20.857860 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:20.857882 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:30:20.858116 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:20.858120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:20.858208 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:20.858318 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:20.858775 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.859578 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.860902 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:20.861357 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:20.861511 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:20.862618 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:20.862627 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:20.862642 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:20.863231 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:20.863281 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:20.863301 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:20.863432 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:20.863569 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 06:30:20.863752 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:30:20.864698 4843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:20.932241 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.931709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38281 (local address 127.0.0.1:34308)
0809 06:30:20.931930 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.932057 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 06:30:20.933305 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:20.932915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:38281 (local address 127.0.0.1:34314)
0809 06:30:20.933119 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:20.933199 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 06:30:20.961555 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 4663
I20260809 06:30:20.983870 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 4144
I20260809 06:30:20.985010 4229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:21.096438 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 4144
2026-08-09T06:30:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2878 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:21 dist-test-slave-mjjr krb5kdc[4930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:21 dist-test-slave-mjjr krb5kdc[4930](info): set up 2 sockets
Aug 09 06:30:21 dist-test-slave-mjjr krb5kdc[4930](info): commencing operation
krb5kdc: starting...
W20260809 06:30:22.449723 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:22 dist-test-slave-mjjr krb5kdc[4930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:30:22.746343 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:44751
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:44751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:23.093557 4949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:23.093926 4949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:23.094101 4949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:23.102196 4949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:23.102356 4949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:23.102448 4949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:23.102543 4949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:23.102622 4949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:23.112689 4949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:44751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:44751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:23.114051 4949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:23.115449 4949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:23.127657 4954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:23.127710 4957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:23.127729 4955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:23.128587 4949 server_base.cc:1034] running on GCE node
I20260809 06:30:23.129359 4949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:23.130761 4949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:23.132026 4949 hybrid_clock.cc:648] HybridClock initialized: now 1786257023131965 us; error 49 us; skew 500 ppm
I20260809 06:30:23.132416 4949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:23 dist-test-slave-mjjr krb5kdc[4930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257023, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:23.135773 4949 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:30:23.137176 4949 webserver.cc:460] Webserver started at http://127.8.131.62:46883/ using document root <none> and password file <none>
I20260809 06:30:23.137850 4949 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:23.137959 4949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:23.138203 4949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:23.141207 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "08a7f672552349b7957e42873c809895"
format_stamp: "Formatted at 2026-08-09 06:30:23 on dist-test-slave-mjjr"
server_key: "180c64bbf22b3b4cdbe588abd740c5c6"
server_key_iv: "081092443f735acd469bfb470191ad9f"
server_key_version: "encryptionkey@0"
I20260809 06:30:23.141971 4949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "08a7f672552349b7957e42873c809895"
format_stamp: "Formatted at 2026-08-09 06:30:23 on dist-test-slave-mjjr"
server_key: "180c64bbf22b3b4cdbe588abd740c5c6"
server_key_iv: "081092443f735acd469bfb470191ad9f"
server_key_version: "encryptionkey@0"
I20260809 06:30:23.147280 4949 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 06:30:23.150837 4965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:23.152344 4949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:23.152496 4949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "08a7f672552349b7957e42873c809895"
format_stamp: "Formatted at 2026-08-09 06:30:23 on dist-test-slave-mjjr"
server_key: "180c64bbf22b3b4cdbe588abd740c5c6"
server_key_iv: "081092443f735acd469bfb470191ad9f"
server_key_version: "encryptionkey@0"
I20260809 06:30:23.152668 4949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:23.167694 4949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:23.171957 4949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:23.172189 4949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:23.191009 4949 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:44751
I20260809 06:30:23.191066 5038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:44751 every 8 connection(s)
I20260809 06:30:23.192466 4949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:23.194676 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 4949
I20260809 06:30:23.194887 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:23.195351 8716 external_mini_cluster.cc:1428] Setting key 32264e91d8011166f1cfa281fd6aefec
I20260809 06:30:23.196557 5041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 06:30:23 dist-test-slave-mjjr krb5kdc[4930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:23.210371 5041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895: Bootstrap starting.
I20260809 06:30:23.214434 5041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:23.215793 5041 log.cc:824] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:23.219220 5041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895: No bootstrap required, opened a new log
I20260809 06:30:23.220257 5044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:23.197237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39728 (local address 127.8.131.62:44751)
0809 06:30:23.198031 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:23.198045 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:23.198095 (+ 50us) server_negotiation.cc:407] Connection header received
0809 06:30:23.200136 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:23.200161 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:23.200637 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:23.201065 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:23.202437 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:23.204501 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:23.205885 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:23.206971 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:23.207174 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:23.211458 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:23.211468 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:23.211486 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:23.211537 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:23.215670 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:23.216470 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:23.216480 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:23.216491 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:23.216649 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:23.217097 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:23.217100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:23.217126 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:23.217792 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:23.218110 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:23.218481 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 06:30:23.218923 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":326,"threads_started":1}
I20260809 06:30:23.227731 5041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a7f672552349b7957e42873c809895" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44751 } }
I20260809 06:30:23.228045 5041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:23.228120 5041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a7f672552349b7957e42873c809895, State: Initialized, Role: FOLLOWER
I20260809 06:30:23.228624 5041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a7f672552349b7957e42873c809895" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44751 } }
I20260809 06:30:23.228929 5041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:23.229089 5041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:23.229377 5041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:23.231608 5041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a7f672552349b7957e42873c809895" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44751 } }
I20260809 06:30:23.232082 5041 leader_election.cc:302] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08a7f672552349b7957e42873c809895; no voters:
I20260809 06:30:23.232614 5041 leader_election.cc:288] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:23.232823 5048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:23.234035 5048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [term 1 LEADER]: Becoming Leader. State: Replica: 08a7f672552349b7957e42873c809895, State: Running, Role: LEADER
I20260809 06:30:23.234532 5048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a7f672552349b7957e42873c809895" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44751 } }
I20260809 06:30:23.234743 5041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:23.237772 5049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08a7f672552349b7957e42873c809895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a7f672552349b7957e42873c809895" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44751 } } }
I20260809 06:30:23.237807 5050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08a7f672552349b7957e42873c809895. Latest consensus state: current_term: 1 leader_uuid: "08a7f672552349b7957e42873c809895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a7f672552349b7957e42873c809895" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 44751 } } }
I20260809 06:30:23.238165 5049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:23.238210 5050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:23.240674 5059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:23.244760 5059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:23.253016 5059 catalog_manager.cc:1347] Generated new cluster ID: 142efce93cf94bd1bb2a701309cfd277
I20260809 06:30:23.253122 5059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:23.271530 5059 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:23.273355 5059 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:23.285531 5059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895: Generated new TSK 0
I20260809 06:30:23.286422 5059 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.131.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
Entry for principal kudu/127.8.131.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab.
I20260809 06:30:23.358999 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44751
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:23.705440 5072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:23.705713 5072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:23.705850 5072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:23.713048 5072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:23.713145 5072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:23.713307 5072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:30:23.724090 5072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.1.keytab
--principal=kudu/127.8.131.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:23.725297 5072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:23.726841 5072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:23.740161 5079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:23.740192 5080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:23.740427 5082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:23.741078 5072 server_base.cc:1034] running on GCE node
I20260809 06:30:23.741736 5072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:23.742645 5072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:23.743870 5072 hybrid_clock.cc:648] HybridClock initialized: now 1786257023743836 us; error 42 us; skew 500 ppm
I20260809 06:30:23.744194 5072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:23 dist-test-slave-mjjr krb5kdc[4930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257023, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:23.747591 5072 init.cc:377] Logged in from keytab as kudu/127.8.131.1@KRBTEST.COM (short username kudu)
I20260809 06:30:23.749070 5072 webserver.cc:460] Webserver started at http://127.8.131.1:41437/ using document root <none> and password file <none>
I20260809 06:30:23.749750 5072 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:23.749899 5072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:23.750209 5072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:23.753118 5072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "f5816ca731554004b8d64a221106c4e1"
format_stamp: "Formatted at 2026-08-09 06:30:23 on dist-test-slave-mjjr"
server_key: "b4dd86c2a9fa320d922f779846ab4376"
server_key_iv: "e7ba59c430be28b8673cd6402d80d87a"
server_key_version: "encryptionkey@0"
I20260809 06:30:23.754014 5072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "f5816ca731554004b8d64a221106c4e1"
format_stamp: "Formatted at 2026-08-09 06:30:23 on dist-test-slave-mjjr"
server_key: "b4dd86c2a9fa320d922f779846ab4376"
server_key_iv: "e7ba59c430be28b8673cd6402d80d87a"
server_key_version: "encryptionkey@0"
I20260809 06:30:23.759342 5072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 06:30:23.762853 5095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:23.764217 5072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:23.764413 5072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "f5816ca731554004b8d64a221106c4e1"
format_stamp: "Formatted at 2026-08-09 06:30:23 on dist-test-slave-mjjr"
server_key: "b4dd86c2a9fa320d922f779846ab4376"
server_key_iv: "e7ba59c430be28b8673cd6402d80d87a"
server_key_version: "encryptionkey@0"
I20260809 06:30:23.764621 5072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:23.786572 5072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:23.791106 5072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:23.791412 5072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:23.792472 5072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:23.793805 5072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:23.793917 5072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:23.794075 5072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:23.794173 5072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:23.828116 5072 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:43005
I20260809 06:30:23.828204 5263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:43005 every 8 connection(s)
I20260809 06:30:23.829430 5072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:30:23.839164 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 5072
I20260809 06:30:23.839352 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:30:23.839959 8716 external_mini_cluster.cc:1428] Setting key 9ef7ace883d01827b8055db26c81695c
Aug 09 06:30:23 dist-test-slave-mjjr krb5kdc[4930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257023, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.1@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:23.850778 5266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:23.831939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:47155 (local address 127.8.131.62:44751)
0809 06:30:23.832425 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:23.832429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:23.833200 (+ 771us) server_negotiation.cc:407] Connection header received
0809 06:30:23.835167 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:23.835172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:23.835243 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:23.835343 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:23.836969 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:23.837731 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:23.839810 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:23.840287 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:23.840393 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:23.844527 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:23.844533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:23.844536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:23.844571 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:23.847682 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:23.848444 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:23.848450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:23.848460 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:23.848579 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:23.849022 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:23.849027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:23.849031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:23.849322 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:23.849491 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:23.850177 (+ 686us) server_negotiation.cc:299] Negotiation successful
0809 06:30:23.850401 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":259,"threads_started":1}
I20260809 06:30:23.851796 5267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:23.832357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:44751 (local address 127.8.131.1:47155)
0809 06:30:23.833028 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:23.833074 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:23.834879 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:23.835542 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:23.835549 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:23.836220 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:23.836773 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:23.836790 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:23.837906 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:23.837910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:23.839598 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:23.839615 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:23.840526 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:23.840538 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:23.840795 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:23.841540 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:30:23.841562 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:23.844265 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:23.847846 (+ 3581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:23.847855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:23.847876 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:23.848314 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:23.848700 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:23.848705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:23.848708 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:23.848899 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:23.849451 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:23.849459 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:23.849797 (+ 338us) client_negotiation.cc:769] Sending connection context
0809 06:30:23.850062 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 06:30:23.850587 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":257,"threads_started":1}
I20260809 06:30:23.853452 5264 heartbeater.cc:347] Connected to a master server at 127.8.131.62:44751
I20260809 06:30:23.853771 5264 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:23.854552 5264 heartbeater.cc:511] Master 127.8.131.62:44751 requested a full tablet report, sending...
I20260809 06:30:23.856835 4984 ts_manager.cc:194] Registered new tserver with Master: f5816ca731554004b8d64a221106c4e1 (127.8.131.1:43005)
I20260809 06:30:23.858286 4984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.1@KRBTEST.COM'} at 127.8.131.1:47155
WARNING: no policy specified for kudu/127.8.131.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
Entry for principal kudu/127.8.131.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab.
I20260809 06:30:23.909372 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44751
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:24.245695 5274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:24.245954 5274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:24.246089 5274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:24.253285 5274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:24.253394 5274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:24.253578 5274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:30:24.263284 5274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.2.keytab
--principal=kudu/127.8.131.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:24.264549 5274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:24.265781 5274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:24.278482 5280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:24.278568 5285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:24.278535 5282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:24.279534 5274 server_base.cc:1034] running on GCE node
I20260809 06:30:24.280121 5274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:24.281015 5274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:24.282235 5274 hybrid_clock.cc:648] HybridClock initialized: now 1786257024282192 us; error 39 us; skew 500 ppm
I20260809 06:30:24.282533 5274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:24 dist-test-slave-mjjr krb5kdc[4930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257024, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:24.285717 5274 init.cc:377] Logged in from keytab as kudu/127.8.131.2@KRBTEST.COM (short username kudu)
I20260809 06:30:24.287283 5274 webserver.cc:460] Webserver started at http://127.8.131.2:38943/ using document root <none> and password file <none>
I20260809 06:30:24.288046 5274 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:24.288213 5274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:24.288524 5274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:24.291270 5274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "9f3c0000a6084d90a3fb8158af8ede1b"
format_stamp: "Formatted at 2026-08-09 06:30:24 on dist-test-slave-mjjr"
server_key: "4728accf5b4e52b307889d6ed52e66be"
server_key_iv: "5b5820b4bb59d55eca0207713ec3d23c"
server_key_version: "encryptionkey@0"
I20260809 06:30:24.292073 5274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "9f3c0000a6084d90a3fb8158af8ede1b"
format_stamp: "Formatted at 2026-08-09 06:30:24 on dist-test-slave-mjjr"
server_key: "4728accf5b4e52b307889d6ed52e66be"
server_key_iv: "5b5820b4bb59d55eca0207713ec3d23c"
server_key_version: "encryptionkey@0"
I20260809 06:30:24.297626 5274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:30:24.301185 5298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:24.302510 5274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:30:24.302706 5274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "9f3c0000a6084d90a3fb8158af8ede1b"
format_stamp: "Formatted at 2026-08-09 06:30:24 on dist-test-slave-mjjr"
server_key: "4728accf5b4e52b307889d6ed52e66be"
server_key_iv: "5b5820b4bb59d55eca0207713ec3d23c"
server_key_version: "encryptionkey@0"
I20260809 06:30:24.302914 5274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:24.332984 5274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:24.337675 5274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:24.337980 5274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:24.339064 5274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:24.340535 5274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:24.340608 5274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:24.340698 5274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:24.340751 5274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:24.374429 5274 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:43035
I20260809 06:30:24.374478 5461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:43035 every 8 connection(s)
I20260809 06:30:24.375749 5274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:30:24.380079 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 5274
I20260809 06:30:24.380293 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:30:24.380735 8716 external_mini_cluster.cc:1428] Setting key 6d0286e5716478992da2b744ff044c94
Aug 09 06:30:24 dist-test-slave-mjjr krb5kdc[4930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257024, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.2@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:24.395900 5467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.378499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:58439 (local address 127.8.131.62:44751)
0809 06:30:24.378960 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:24.378965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:24.379776 (+ 811us) server_negotiation.cc:407] Connection header received
0809 06:30:24.381835 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:24.381841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:24.381932 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:24.382053 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:24.383587 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.384361 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.385915 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.386344 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.386489 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:24.390174 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:24.390179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:24.390182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:24.390214 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:24.392882 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:24.393576 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:24.393581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:24.393585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:24.393719 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:24.394149 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:24.394154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:24.394157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:24.394434 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:24.394606 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:24.395266 (+ 660us) server_negotiation.cc:299] Negotiation successful
0809 06:30:24.395501 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":264,"threads_started":1}
I20260809 06:30:24.396814 5468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.378845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:44751 (local address 127.8.131.2:58439)
0809 06:30:24.379510 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:24.379596 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:24.381538 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:24.382288 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:24.382295 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:24.382884 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:24.383415 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:24.383431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.384540 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.384546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:24.385767 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:24.385778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.386586 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.386593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:24.386830 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:24.387515 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:30:24.387538 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:24.389936 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:24.393028 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:24.393037 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:24.393051 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:24.393447 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:24.393830 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:24.393836 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:24.393839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:24.394023 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:24.394578 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:24.394585 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:24.394901 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 06:30:24.395162 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 06:30:24.395677 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":281,"threads_started":1}
I20260809 06:30:24.398268 5462 heartbeater.cc:347] Connected to a master server at 127.8.131.62:44751
I20260809 06:30:24.398563 5462 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:24.399240 5462 heartbeater.cc:511] Master 127.8.131.62:44751 requested a full tablet report, sending...
I20260809 06:30:24.400779 4984 ts_manager.cc:194] Registered new tserver with Master: 9f3c0000a6084d90a3fb8158af8ede1b (127.8.131.2:43035)
I20260809 06:30:24.402120 4984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.2@KRBTEST.COM'} at 127.8.131.2:58439
WARNING: no policy specified for kudu/127.8.131.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
Entry for principal kudu/127.8.131.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab.
I20260809 06:30:24.446310 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44751
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:38103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:24.797601 5471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:24.797863 5471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:24.797955 5471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:24.805037 5471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:24.805145 5471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:24.805263 5471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:30:24.815339 5471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.3.keytab
--principal=kudu/127.8.131.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:44751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:24.816664 5471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:24.817931 5471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:24.831903 5479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:24.831955 5480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:24.832430 5482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:24.833318 5471 server_base.cc:1034] running on GCE node
I20260809 06:30:24.833953 5471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:24.835026 5471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:24.836297 5471 hybrid_clock.cc:648] HybridClock initialized: now 1786257024836246 us; error 65 us; skew 500 ppm
I20260809 06:30:24.836637 5471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:24 dist-test-slave-mjjr krb5kdc[4930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257024, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:24.840102 5471 init.cc:377] Logged in from keytab as kudu/127.8.131.3@KRBTEST.COM (short username kudu)
I20260809 06:30:24.841694 5471 webserver.cc:460] Webserver started at http://127.8.131.3:45137/ using document root <none> and password file <none>
I20260809 06:30:24.842394 5471 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:24.842553 5471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:24.842923 5471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:24.846060 5471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "b1ef1308a7b444f7ac05c6d30751139a"
format_stamp: "Formatted at 2026-08-09 06:30:24 on dist-test-slave-mjjr"
server_key: "f7a7d321a48a362ed9abbf3816a07a82"
server_key_iv: "e92c94221d8188cff4fc5abb71bc3503"
server_key_version: "encryptionkey@0"
I20260809 06:30:24.846933 5471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "b1ef1308a7b444f7ac05c6d30751139a"
format_stamp: "Formatted at 2026-08-09 06:30:24 on dist-test-slave-mjjr"
server_key: "f7a7d321a48a362ed9abbf3816a07a82"
server_key_iv: "e92c94221d8188cff4fc5abb71bc3503"
server_key_version: "encryptionkey@0"
I20260809 06:30:24.852269 5471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 06:30:24.856033 5492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:24.857569 5471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:30:24.857754 5471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "b1ef1308a7b444f7ac05c6d30751139a"
format_stamp: "Formatted at 2026-08-09 06:30:24 on dist-test-slave-mjjr"
server_key: "f7a7d321a48a362ed9abbf3816a07a82"
server_key_iv: "e92c94221d8188cff4fc5abb71bc3503"
server_key_version: "encryptionkey@0"
I20260809 06:30:24.857972 5471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:24.861093 5264 heartbeater.cc:503] Master 127.8.131.62:44751 was elected leader, sending a full tablet report...
I20260809 06:30:24.880628 5471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:24.884732 5471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:24.885056 5471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:24.886037 5471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:24.887426 5471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:24.887502 5471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:24.887688 5471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:24.887753 5471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 06:30:24.922030 5471 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:46113
I20260809 06:30:24.922122 5659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:46113 every 8 connection(s)
I20260809 06:30:24.923354 5471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:30:24.927102 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 5471
I20260809 06:30:24.927306 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:30:24.927861 8716 external_mini_cluster.cc:1428] Setting key dd8df90b8ea01c04f38195123c8a50a8
Aug 09 06:30:24 dist-test-slave-mjjr krb5kdc[4930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257024, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.3@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:24.943773 5665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.926061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:33305 (local address 127.8.131.62:44751)
0809 06:30:24.926546 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:24.926551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:24.927731 (+ 1180us) server_negotiation.cc:407] Connection header received
0809 06:30:24.929732 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:24.929737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:24.929834 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:24.929923 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:24.931538 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.932396 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.933848 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.934240 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.934339 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:24.937816 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:24.937820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:24.937823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:24.937857 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:24.940664 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:24.941359 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:24.941364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:24.941367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:24.941526 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:24.942019 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:24.942023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:24.942026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:24.942396 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:24.942561 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:24.943263 (+ 702us) server_negotiation.cc:299] Negotiation successful
0809 06:30:24.943439 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":235,"threads_started":1}
I20260809 06:30:24.944746 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.926600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:44751 (local address 127.8.131.3:33305)
0809 06:30:24.927527 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:24.927572 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:24.929478 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:24.930197 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:24.930209 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:24.930844 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:24.931378 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:24.931400 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.932534 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.932537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:24.933716 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:24.933727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.934417 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.934422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:24.934604 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:24.935296 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:30:24.935318 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:24.937608 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:24.940812 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:24.940819 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:24.940830 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:24.941239 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:24.941641 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:30:24.941649 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:30:24.941653 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:30:24.941897 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:30:24.942532 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:24.942540 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:24.942889 (+ 349us) client_negotiation.cc:769] Sending connection context
0809 06:30:24.943167 (+ 278us) client_negotiation.cc:240] Negotiation successful
0809 06:30:24.943577 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":466,"threads_started":1}
I20260809 06:30:24.946094 5663 heartbeater.cc:347] Connected to a master server at 127.8.131.62:44751
I20260809 06:30:24.946434 5663 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:24.947144 5663 heartbeater.cc:511] Master 127.8.131.62:44751 requested a full tablet report, sending...
I20260809 06:30:24.948597 4984 ts_manager.cc:194] Registered new tserver with Master: b1ef1308a7b444f7ac05c6d30751139a (127.8.131.3:46113)
I20260809 06:30:24.949427 4984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.131.3@KRBTEST.COM'} at 127.8.131.3:33305
I20260809 06:30:24.955552 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:30:24.974655 5665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.960315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39734 (local address 127.8.131.62:44751)
0809 06:30:24.960488 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:24.960492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:24.960901 (+ 409us) server_negotiation.cc:407] Connection header received
0809 06:30:24.961045 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:24.961049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:24.961116 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:24.961255 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:24.962928 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.963831 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.965256 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:24.965725 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:24.965879 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:24.968638 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:24.968644 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:24.968647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:24.968681 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:24.971597 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:24.972445 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:24.972451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:24.972455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:24.972608 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:24.973047 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:24.973059 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:24.973063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:24.973434 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:24.973702 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:24.974085 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 06:30:24.974325 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:30:24.978665 4984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:30:24.985459 4984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:30:25.034730 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.997575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.8.131.2:43035)
0809 06:30:25.024340 (+ 26765us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:25.024352 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:25.024392 (+ 40us) server_negotiation.cc:407] Connection header received
0809 06:30:25.024520 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:25.024531 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:25.024868 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:25.025109 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:25.026549 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.028504 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.032072 (+ 3568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.033519 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.033643 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.033840 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:25.034212 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 06:30:25.034374 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26541,"thread_start_us":28664,"threads_started":1}
I20260809 06:30:25.034886 5680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.997638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:46113 (local address 127.0.0.1:55796)
0809 06:30:25.025691 (+ 28053us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.025710 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:25.025844 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:25.026968 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:25.026971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:25.026991 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:25.027318 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.027325 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.029168 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.029176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.031905 (+ 2729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.031919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.033985 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.033991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.034167 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.034195 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 06:30:25.034285 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 06:30:25.034390 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27917,"mutex_wait_us":123,"thread_start_us":27752,"threads_started":1}
I20260809 06:30:25.034888 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.997291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:43035 (local address 127.0.0.1:46306)
0809 06:30:25.023100 (+ 25809us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.023171 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:25.023454 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:25.025463 (+ 2009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:25.025479 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:25.025528 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:25.026071 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.026087 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.028881 (+ 2794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.028887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.031904 (+ 3017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.031919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.033695 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.033701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.033932 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.033998 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 06:30:25.034135 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 06:30:25.034326 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25409,"mutex_wait_us":145,"thread_start_us":30360,"threads_started":1}
I20260809 06:30:25.035571 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.997913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.0.0.1:57768)
0809 06:30:25.027952 (+ 30039us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.027980 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:25.028110 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:25.028696 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:25.028701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:25.028730 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:25.029253 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.029266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.030827 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.030833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.032564 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.032578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.034623 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.034628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.034836 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.034889 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 06:30:25.034976 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 06:30:25.035147 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29920,"mutex_wait_us":33,"thread_start_us":24616,"threads_started":1}
I20260809 06:30:25.035892 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.998394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.8.131.3:46113)
0809 06:30:25.022044 (+ 23650us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:25.022059 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:25.025988 (+ 3929us) server_negotiation.cc:407] Connection header received
0809 06:30:25.026090 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:25.026096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:25.026333 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:25.026537 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:25.027532 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.028985 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.032101 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.033452 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.033580 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.033844 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:25.035387 (+ 1543us) server_negotiation.cc:299] Negotiation successful
0809 06:30:25.035521 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23350,"thread_start_us":23683,"threads_started":1}
I20260809 06:30:25.036438 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:24.998650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.8.131.1:43005)
0809 06:30:25.024124 (+ 25474us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:25.024137 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:25.028107 (+ 3970us) server_negotiation.cc:407] Connection header received
0809 06:30:25.028244 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:25.028250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:25.028447 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:25.028654 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:25.029478 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.030683 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.032749 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.034126 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.034252 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.034469 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:25.035923 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0809 06:30:25.036083 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25233,"thread_start_us":28092,"threads_started":1}
I20260809 06:30:25.039580 5167 tablet_service.cc:1467] Processing CreateTablet for tablet 0844e8dab0b34158875c7aca7dc87a80 (DEFAULT_TABLE table=test-table [id=92bbcefe30e64771b9ba450a11926761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:25.039574 5365 tablet_service.cc:1467] Processing CreateTablet for tablet 0844e8dab0b34158875c7aca7dc87a80 (DEFAULT_TABLE table=test-table [id=92bbcefe30e64771b9ba450a11926761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:25.039574 5564 tablet_service.cc:1467] Processing CreateTablet for tablet 0844e8dab0b34158875c7aca7dc87a80 (DEFAULT_TABLE table=test-table [id=92bbcefe30e64771b9ba450a11926761]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:25.040902 5167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0844e8dab0b34158875c7aca7dc87a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:25.040971 5365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0844e8dab0b34158875c7aca7dc87a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:25.040971 5564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0844e8dab0b34158875c7aca7dc87a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:25.052403 5685 tablet_bootstrap.cc:492] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1: Bootstrap starting.
I20260809 06:30:25.056231 5686 tablet_bootstrap.cc:492] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: Bootstrap starting.
I20260809 06:30:25.056423 5687 tablet_bootstrap.cc:492] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: Bootstrap starting.
I20260809 06:30:25.056934 5685 tablet_bootstrap.cc:654] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:25.058391 5685 log.cc:824] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:25.060825 5686 tablet_bootstrap.cc:654] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:25.061455 5685 tablet_bootstrap.cc:492] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1: No bootstrap required, opened a new log
I20260809 06:30:25.061741 5685 ts_tablet_manager.cc:1397] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 06:30:25.062212 5686 log.cc:824] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:25.062749 5687 tablet_bootstrap.cc:654] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:25.063930 5687 log.cc:824] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:25.065743 5686 tablet_bootstrap.cc:492] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: No bootstrap required, opened a new log
I20260809 06:30:25.066082 5686 ts_tablet_manager.cc:1397] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260809 06:30:25.070379 5685 raft_consensus.cc:348] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } }
I20260809 06:30:25.070741 5685 raft_consensus.cc:374] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:25.070885 5685 raft_consensus.cc:729] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5816ca731554004b8d64a221106c4e1, State: Initialized, Role: FOLLOWER
I20260809 06:30:25.071509 5685 consensus_queue.cc:260] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } }
I20260809 06:30:25.073339 5685 ts_tablet_manager.cc:1428] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1: Time spent starting tablet: real 0.011s user 0.000s sys 0.010s
I20260809 06:30:25.078498 5687 tablet_bootstrap.cc:492] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: No bootstrap required, opened a new log
I20260809 06:30:25.078820 5687 ts_tablet_manager.cc:1397] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.007s
I20260809 06:30:25.079564 5686 raft_consensus.cc:348] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } }
I20260809 06:30:25.080004 5686 raft_consensus.cc:374] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:25.080109 5686 raft_consensus.cc:729] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ef1308a7b444f7ac05c6d30751139a, State: Initialized, Role: FOLLOWER
I20260809 06:30:25.080632 5686 consensus_queue.cc:260] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } }
I20260809 06:30:25.081784 5663 heartbeater.cc:503] Master 127.8.131.62:44751 was elected leader, sending a full tablet report...
W20260809 06:30:25.081792 5265 tablet.cc:2369] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:30:25.082257 5686 ts_tablet_manager.cc:1428] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260809 06:30:25.087956 5687 raft_consensus.cc:348] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } }
I20260809 06:30:25.088351 5687 raft_consensus.cc:374] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:25.088481 5687 raft_consensus.cc:729] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3c0000a6084d90a3fb8158af8ede1b, State: Initialized, Role: FOLLOWER
I20260809 06:30:25.089161 5687 consensus_queue.cc:260] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } }
I20260809 06:30:25.090220 5462 heartbeater.cc:503] Master 127.8.131.62:44751 was elected leader, sending a full tablet report...
I20260809 06:30:25.090700 5687 ts_tablet_manager.cc:1428] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260809 06:30:25.128005 5465 tablet.cc:2369] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 06:30:25.175078 5664 tablet.cc:2369] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:30:25.223949 5697 raft_consensus.cc:482] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:30:25.224247 5697 raft_consensus.cc:504] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } }
I20260809 06:30:25.226096 5697 leader_election.cc:288] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1ef1308a7b444f7ac05c6d30751139a (127.8.131.3:46113), f5816ca731554004b8d64a221106c4e1 (127.8.131.1:43005)
I20260809 06:30:25.233217 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.226575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:49405 (local address 127.8.131.3:46113)
0809 06:30:25.226793 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:25.226801 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:25.227681 (+ 880us) server_negotiation.cc:407] Connection header received
0809 06:30:25.227877 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:25.227882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:25.227966 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:25.228067 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:25.228700 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.229489 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.231399 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.232244 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.232342 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.232433 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:25.232826 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 06:30:25.232955 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:30:25.233306 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.226561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:46113 (local address 127.8.131.2:49405)
0809 06:30:25.227536 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.227555 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:25.227732 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:25.228093 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:25.228098 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:25.228122 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:25.228550 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.228561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.229632 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.229640 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.231247 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.231259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.232481 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.232488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.232653 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.232692 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 06:30:25.232786 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 06:30:25.232930 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":47,"thread_start_us":531,"threads_started":1}
I20260809 06:30:25.233567 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.226639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:42441 (local address 127.8.131.1:43005)
0809 06:30:25.226899 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:25.226908 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:25.227872 (+ 964us) server_negotiation.cc:407] Connection header received
0809 06:30:25.228054 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:25.228062 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:25.228140 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:25.228246 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:25.228881 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.229593 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.231486 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.232582 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.232712 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.232826 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:25.233162 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 06:30:25.233305 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 06:30:25.233719 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.226574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.8.131.2:42441)
0809 06:30:25.227727 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.227756 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:25.227900 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:25.228277 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:25.228280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:25.228306 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:25.228705 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.228716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.229746 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.229750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.231306 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:25.231341 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.232813 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.232819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:25.232988 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.233021 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 06:30:25.233119 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 06:30:25.233245 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":62,"spinlock_wait_cycles":16640,"thread_start_us":622,"threads_started":1}
I20260809 06:30:25.234642 5597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0844e8dab0b34158875c7aca7dc87a80" candidate_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ef1308a7b444f7ac05c6d30751139a" is_pre_election: true
I20260809 06:30:25.234776 5203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0844e8dab0b34158875c7aca7dc87a80" candidate_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5816ca731554004b8d64a221106c4e1" is_pre_election: true
I20260809 06:30:25.235210 5597 raft_consensus.cc:2454] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3c0000a6084d90a3fb8158af8ede1b in term 0.
I20260809 06:30:25.235288 5203 raft_consensus.cc:2454] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3c0000a6084d90a3fb8158af8ede1b in term 0.
I20260809 06:30:25.235989 5301 leader_election.cc:302] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3c0000a6084d90a3fb8158af8ede1b, b1ef1308a7b444f7ac05c6d30751139a; no voters:
I20260809 06:30:25.236379 5697 raft_consensus.cc:2781] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:30:25.236536 5697 raft_consensus.cc:482] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:30:25.236656 5697 raft_consensus.cc:3037] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:25.238901 5697 raft_consensus.cc:504] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } }
I20260809 06:30:25.239517 5697 leader_election.cc:288] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [CANDIDATE]: Term 1 election: Requested vote from peers b1ef1308a7b444f7ac05c6d30751139a (127.8.131.3:46113), f5816ca731554004b8d64a221106c4e1 (127.8.131.1:43005)
I20260809 06:30:25.240037 5203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0844e8dab0b34158875c7aca7dc87a80" candidate_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5816ca731554004b8d64a221106c4e1"
I20260809 06:30:25.240058 5597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0844e8dab0b34158875c7aca7dc87a80" candidate_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ef1308a7b444f7ac05c6d30751139a"
I20260809 06:30:25.240257 5597 raft_consensus.cc:3037] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:25.240226 5203 raft_consensus.cc:3037] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:25.242555 5203 raft_consensus.cc:2454] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3c0000a6084d90a3fb8158af8ede1b in term 1.
I20260809 06:30:25.242568 5597 raft_consensus.cc:2454] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3c0000a6084d90a3fb8158af8ede1b in term 1.
I20260809 06:30:25.243001 5302 leader_election.cc:302] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3c0000a6084d90a3fb8158af8ede1b, f5816ca731554004b8d64a221106c4e1; no voters:
I20260809 06:30:25.243290 5697 raft_consensus.cc:2781] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:25.243816 5697 raft_consensus.cc:686] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 1 LEADER]: Becoming Leader. State: Replica: 9f3c0000a6084d90a3fb8158af8ede1b, State: Running, Role: LEADER
I20260809 06:30:25.244272 5697 consensus_queue.cc:237] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } }
I20260809 06:30:25.249325 4984 catalog_manager.cc:5495] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f3c0000a6084d90a3fb8158af8ede1b (127.8.131.2). New cstate: current_term: 1 leader_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3c0000a6084d90a3fb8158af8ede1b" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 43035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 } health_report { overall_health: UNKNOWN } } }
I20260809 06:30:25.274391 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.268647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.8.131.2:43035)
0809 06:30:25.268886 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:25.268891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:25.269049 (+ 158us) server_negotiation.cc:407] Connection header received
0809 06:30:25.269249 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:25.269253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:25.269331 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:25.269468 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:30:25.269913 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.270596 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.272302 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.272679 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.272785 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.273325 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:30:25.273840 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:30:25.273942 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:25.274064 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 06:30:25.274123 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 06:30:25.282763 5597 raft_consensus.cc:1261] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [term 1 FOLLOWER]: Refusing update from remote peer 9f3c0000a6084d90a3fb8158af8ede1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:25.282763 5203 raft_consensus.cc:1261] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [term 1 FOLLOWER]: Refusing update from remote peer 9f3c0000a6084d90a3fb8158af8ede1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:25.283725 5700 consensus_queue.cc:1035] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1ef1308a7b444f7ac05c6d30751139a" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:25.284085 5697 consensus_queue.cc:1035] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5816ca731554004b8d64a221106c4e1" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 43005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:25.295953 5703 mvcc.cc:205] Tried to move back new op lower bound from 7316508775548035072 to 7316508775404212224. Current Snapshot: MvccSnapshot[applied={T|T < 7316508775548035072}]
I20260809 06:30:25.301319 5705 mvcc.cc:205] Tried to move back new op lower bound from 7316508775548035072 to 7316508775404212224. Current Snapshot: MvccSnapshot[applied={T|T < 7316508775548035072}]
I20260809 06:30:25.309890 4984 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:25.310173 4984 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:25.313570 4984 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08a7f672552349b7957e42873c809895: Sending DeleteTablet for 3 replicas of tablet 0844e8dab0b34158875c7aca7dc87a80
I20260809 06:30:25.314844 5365 tablet_service.cc:1514] Processing DeleteTablet for tablet 0844e8dab0b34158875c7aca7dc87a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:25 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:46306
I20260809 06:30:25.314891 5564 tablet_service.cc:1514] Processing DeleteTablet for tablet 0844e8dab0b34158875c7aca7dc87a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:25 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:55796
I20260809 06:30:25.315033 5167 tablet_service.cc:1514] Processing DeleteTablet for tablet 0844e8dab0b34158875c7aca7dc87a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:25 UTC) from {username='kudu', principal='kudu/127.8.131.62@KRBTEST.COM'} at 127.0.0.1:57768
I20260809 06:30:25.317298 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 5072
I20260809 06:30:25.347904 5717 tablet_replica.cc:330] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: stopping tablet replica
I20260809 06:30:25.351704 5717 raft_consensus.cc:2229] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:30:25.352792 5717 raft_consensus.cc:2258] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:25.358829 5717 ts_tablet_manager.cc:1905] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:25.360213 5718 tablet_replica.cc:330] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: stopping tablet replica
I20260809 06:30:25.360445 5719 tablet_replica.cc:330] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1: stopping tablet replica
I20260809 06:30:25.360803 5718 raft_consensus.cc:2229] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:25.360906 5719 raft_consensus.cc:2229] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:25.361459 5718 raft_consensus.cc:2258] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 06:30:25 dist-test-slave-mjjr krb5kdc[4930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.1@KRBTEST.COM
I20260809 06:30:25.364136 5718 ts_tablet_manager.cc:1905] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:25.365571 5719 raft_consensus.cc:2258] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:25.369959 5717 ts_tablet_manager.cc:1918] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:25.370200 5717 log.cc:1196] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/0844e8dab0b34158875c7aca7dc87a80
I20260809 06:30:25.371171 5717 ts_tablet_manager.cc:1939] T 0844e8dab0b34158875c7aca7dc87a80 P 9f3c0000a6084d90a3fb8158af8ede1b: Deleting consensus metadata
I20260809 06:30:25.373171 4970 catalog_manager.cc:4841] TS 9f3c0000a6084d90a3fb8158af8ede1b (127.8.131.2:43035): tablet 0844e8dab0b34158875c7aca7dc87a80 (table test-table [id=92bbcefe30e64771b9ba450a11926761]) successfully deleted
I20260809 06:30:25.374286 5719 ts_tablet_manager.cc:1905] T 0844e8dab0b34158875c7aca7dc87a80 P f5816ca731554004b8d64a221106c4e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:25.376152 5718 ts_tablet_manager.cc:1918] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:25.376397 5718 log.cc:1196] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/0844e8dab0b34158875c7aca7dc87a80
I20260809 06:30:25.376981 5718 ts_tablet_manager.cc:1939] T 0844e8dab0b34158875c7aca7dc87a80 P b1ef1308a7b444f7ac05c6d30751139a: Deleting consensus metadata
I20260809 06:30:25.376930 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.318946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.8.131.1:43005)
0809 06:30:25.319177 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:25.319185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:25.343863 (+ 24678us) server_negotiation.cc:407] Connection header received
0809 06:30:25.344512 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:25.344521 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:25.344634 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:25.344728 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:25.348218 (+ 3490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.349804 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.354851 (+ 5047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.355774 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.355917 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.364650 (+ 8733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:25.364667 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:25.364687 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:25.364735 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:25.369648 (+ 4913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:25.370884 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:25.370893 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:25.370899 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:25.371110 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:25.374770 (+ 3660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:25.374776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:25.374781 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:25.375257 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:25.375553 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:25.376186 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 06:30:25.376483 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:30:25.378556 5259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:25.386011 4968 catalog_manager.cc:4841] TS b1ef1308a7b444f7ac05c6d30751139a (127.8.131.3:46113): tablet 0844e8dab0b34158875c7aca7dc87a80 (table test-table [id=92bbcefe30e64771b9ba450a11926761]) successfully deleted
I20260809 06:30:25.508877 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 5072
W20260809 06:30:25.540241 4969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.131.1:43005 (error 108)
I20260809 06:30:25.541083 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 5274
I20260809 06:30:25.542402 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.541529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.0.0.1:58538)
0809 06:30:25.542044 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.542178 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":272,"threads_started":1}
W20260809 06:30:25.542732 4969 catalog_manager.cc:4584] TS f5816ca731554004b8d64a221106c4e1 (127.8.131.1:43005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0844e8dab0b34158875c7aca7dc87a80: Network error: Client connection negotiation failed: client connection to 127.8.131.1:43005: connect: Connection refused (error 111)
Aug 09 06:30:25 dist-test-slave-mjjr krb5kdc[4930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.2@KRBTEST.COM
I20260809 06:30:25.559861 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.542093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47800 (local address 127.8.131.2:43035)
0809 06:30:25.542273 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:25.542278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:25.542297 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:30:25.542482 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:25.542489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:25.542581 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:25.542701 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:25.544462 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.545291 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.547191 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.547620 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.547901 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.552866 (+ 4965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:25.552882 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:25.552902 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:25.552959 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:25.556937 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:25.557646 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:25.557658 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:25.557666 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:25.557824 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:25.558184 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:25.558189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:25.558192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:25.558633 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:25.558996 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:25.559197 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 06:30:25.559444 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 06:30:25.560611 5456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:25.596745 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.596196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.0.0.1:58542)
0809 06:30:25.596446 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.596560 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 06:30:25.597873 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.597432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.0.0.1:58550)
0809 06:30:25.597600 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.597687 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 06:30:25.658526 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.657921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.0.0.1:58562)
0809 06:30:25.658215 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.658336 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260809 06:30:25.659747 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.659322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.0.0.1:58572)
0809 06:30:25.659461 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.659530 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 06:30:25.677381 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 5274
I20260809 06:30:25.703815 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 5471
Aug 09 06:30:25 dist-test-slave-mjjr krb5kdc[4930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.3@KRBTEST.COM
I20260809 06:30:25.719197 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.704650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42596 (local address 127.8.131.3:46113)
0809 06:30:25.704881 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:25.704885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:25.704908 (+ 23us) server_negotiation.cc:407] Connection header received
0809 06:30:25.705042 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:25.705046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:25.705117 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:25.705226 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:25.706561 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.707347 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.708672 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:25.709102 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:25.709223 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:25.713653 (+ 4430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:25.713668 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:25.713684 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:25.713722 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:25.716738 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:25.717370 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:25.717378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:25.717381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:25.717514 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:25.717838 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:25.717841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:25.717843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:25.718175 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:25.718404 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:25.718627 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 06:30:25.718799 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260809 06:30:25.720062 5652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:25.740640 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.740188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.0.0.1:58582)
0809 06:30:25.740398 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.740485 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 06:30:25.741706 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.741288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.0.0.1:58594)
0809 06:30:25.741493 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.741564 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 06:30:25.809664 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 5471
I20260809 06:30:25.828141 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 4949
I20260809 06:30:25.829298 5029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:25.949970 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:25.949456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:43005 (local address 127.0.0.1:58606)
0809 06:30:25.949679 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:25.949773 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.1:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 06:30:25.950186 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 4949
2026-08-09T06:30:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4853 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T06:30:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:26Z Disabled control of system clock
I20260809 06:30:26.042526 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:33129
--webserver_interface=127.8.131.62
--webserver_port=0
--builtin_ntp_servers=127.8.131.20:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:33129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 06:30:26.383616 5737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:26.384001 5737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:26.384151 5737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:26.391423 5737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:26.391551 5737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:26.391628 5737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:26.391768 5737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:26.391834 5737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:26.400964 5737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:33129
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:33129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:26.402220 5737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:26.404119 5737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:26.415937 5746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:26.415905 5751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:26.415997 5747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:26.416966 5737 server_base.cc:1034] running on GCE node
I20260809 06:30:26.417719 5737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:26.419080 5737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:26.420297 5737 hybrid_clock.cc:648] HybridClock initialized: now 1786257026420252 us; error 42 us; skew 500 ppm
I20260809 06:30:26.420673 5737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:26.427956 5737 webserver.cc:460] Webserver started at http://127.8.131.62:35497/ using document root <none> and password file <none>
I20260809 06:30:26.428659 5737 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:26.428785 5737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:26.429131 5737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:26.432080 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "cc69583d081a41e58850e812415ce8b7"
format_stamp: "Formatted at 2026-08-09 06:30:26 on dist-test-slave-mjjr"
server_key: "daa6d32da0bcc0e949771b5ed14c10e3"
server_key_iv: "b48ec467bfd630677125e65c7075a933"
server_key_version: "encryptionkey@0"
I20260809 06:30:26.432906 5737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "cc69583d081a41e58850e812415ce8b7"
format_stamp: "Formatted at 2026-08-09 06:30:26 on dist-test-slave-mjjr"
server_key: "daa6d32da0bcc0e949771b5ed14c10e3"
server_key_iv: "b48ec467bfd630677125e65c7075a933"
server_key_version: "encryptionkey@0"
I20260809 06:30:26.437955 5737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:30:26.441506 5758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:26.442847 5737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:30:26.443068 5737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "cc69583d081a41e58850e812415ce8b7"
format_stamp: "Formatted at 2026-08-09 06:30:26 on dist-test-slave-mjjr"
server_key: "daa6d32da0bcc0e949771b5ed14c10e3"
server_key_iv: "b48ec467bfd630677125e65c7075a933"
server_key_version: "encryptionkey@0"
I20260809 06:30:26.443300 5737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:26.464686 5737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:26.465523 5737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:26.465828 5737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:26.484624 5737 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:33129
I20260809 06:30:26.484643 5827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:33129 every 8 connection(s)
I20260809 06:30:26.485993 5737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:26.489164 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 5737
I20260809 06:30:26.489362 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:26.489944 8716 external_mini_cluster.cc:1428] Setting key f08cf9078a96eac3635d3174fb663ac9
I20260809 06:30:26.490154 5829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:26.504000 5833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:26.492013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.8.131.62:33129)
0809 06:30:26.493027 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:26.493040 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:26.493088 (+ 48us) server_negotiation.cc:407] Connection header received
0809 06:30:26.494889 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:26.494909 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:26.495337 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:26.495754 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:26.496311 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:26.497620 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:26.499491 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:26.500244 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:26.500376 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:26.501664 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:26.501671 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:26.501684 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:26.502273 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:26.502348 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:26.502370 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:26.502467 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:26.502689 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 06:30:26.503024 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":381,"threads_started":1}
I20260809 06:30:26.504446 5829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7: Bootstrap starting.
I20260809 06:30:26.507788 5829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:26.508749 5829 log.cc:824] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:26.511471 5829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7: No bootstrap required, opened a new log
I20260809 06:30:26.518625 5829 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69583d081a41e58850e812415ce8b7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33129 } }
I20260809 06:30:26.518947 5829 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:26.519019 5829 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc69583d081a41e58850e812415ce8b7, State: Initialized, Role: FOLLOWER
I20260809 06:30:26.519484 5829 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69583d081a41e58850e812415ce8b7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33129 } }
I20260809 06:30:26.519747 5829 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:26.519857 5829 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:26.519979 5829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:26.521977 5829 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69583d081a41e58850e812415ce8b7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33129 } }
I20260809 06:30:26.522409 5829 leader_election.cc:302] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc69583d081a41e58850e812415ce8b7; no voters:
I20260809 06:30:26.523017 5829 leader_election.cc:288] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:26.523180 5837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:26.524104 5837 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [term 1 LEADER]: Becoming Leader. State: Replica: cc69583d081a41e58850e812415ce8b7, State: Running, Role: LEADER
I20260809 06:30:26.524591 5837 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69583d081a41e58850e812415ce8b7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33129 } }
I20260809 06:30:26.525337 5829 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:26.527835 5840 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc69583d081a41e58850e812415ce8b7. Latest consensus state: current_term: 1 leader_uuid: "cc69583d081a41e58850e812415ce8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69583d081a41e58850e812415ce8b7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33129 } } }
I20260809 06:30:26.528307 5840 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:26.528954 5838 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc69583d081a41e58850e812415ce8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69583d081a41e58850e812415ce8b7" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 33129 } } }
I20260809 06:30:26.529264 5838 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:26.532341 5854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:26.536741 5854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:26.544565 5854 catalog_manager.cc:1347] Generated new cluster ID: dd8f12a89ba542d1827fdefc97681e10
I20260809 06:30:26.544678 5854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:26.559527 5854 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:26.561203 5854 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:26.580459 5854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7: Generated new TSK 0
I20260809 06:30:26.581574 5854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:26.590481 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.1:0
--local_ip_for_outbound_sockets=127.8.131.1
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33129
--builtin_ntp_servers=127.8.131.20:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 06:30:26.934535 5871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:26.934808 5871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:26.934893 5871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:26.942083 5871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:26.942183 5871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:26.942288 5871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.1
I20260809 06:30:26.951730 5871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.131.1
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.1
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:26.952996 5871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:26.954672 5871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:26.968279 5878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:26.968237 5880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:26.968325 5877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:26.969237 5871 server_base.cc:1034] running on GCE node
I20260809 06:30:26.969826 5871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:26.970711 5871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:26.971911 5871 hybrid_clock.cc:648] HybridClock initialized: now 1786257026971877 us; error 35 us; skew 500 ppm
I20260809 06:30:26.972177 5871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:26.974282 5871 webserver.cc:460] Webserver started at http://127.8.131.1:44297/ using document root <none> and password file <none>
I20260809 06:30:26.974931 5871 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:26.975016 5871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:26.975291 5871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:26.978135 5871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/data/instance:
uuid: "a0064f94232c4d8d8a6e0da532e793a3"
format_stamp: "Formatted at 2026-08-09 06:30:26 on dist-test-slave-mjjr"
server_key: "aca5964e4914a646524e299401243446"
server_key_iv: "f5b552f60da40d7a66db42ebf3e00765"
server_key_version: "encryptionkey@0"
I20260809 06:30:26.978881 5871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance:
uuid: "a0064f94232c4d8d8a6e0da532e793a3"
format_stamp: "Formatted at 2026-08-09 06:30:26 on dist-test-slave-mjjr"
server_key: "aca5964e4914a646524e299401243446"
server_key_iv: "f5b552f60da40d7a66db42ebf3e00765"
server_key_version: "encryptionkey@0"
I20260809 06:30:26.983911 5871 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:30:26.987264 5891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:26.988722 5871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:26.988886 5871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/wal
uuid: "a0064f94232c4d8d8a6e0da532e793a3"
format_stamp: "Formatted at 2026-08-09 06:30:26 on dist-test-slave-mjjr"
server_key: "aca5964e4914a646524e299401243446"
server_key_iv: "f5b552f60da40d7a66db42ebf3e00765"
server_key_version: "encryptionkey@0"
I20260809 06:30:26.989065 5871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:27.004117 5871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:27.004828 5871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:27.005052 5871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:27.005939 5871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:27.007304 5871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:27.007391 5871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:27.007470 5871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:27.007529 5871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:27.042501 5871 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.1:41525
I20260809 06:30:27.042565 6060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.1:41525 every 8 connection(s)
I20260809 06:30:27.043916 5871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/data/info.pb
I20260809 06:30:27.051914 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 5871
I20260809 06:30:27.052088 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/wal/instance
I20260809 06:30:27.052635 8716 external_mini_cluster.cc:1428] Setting key 868fbc64633e8c6c786403be2b0e1e6c
I20260809 06:30:27.055459 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.2:0
--local_ip_for_outbound_sockets=127.8.131.2
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33129
--builtin_ntp_servers=127.8.131.20:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 06:30:27.057191 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:27.046644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.1:39247 (local address 127.8.131.62:33129)
0809 06:30:27.047151 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:27.047156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:27.048293 (+ 1137us) server_negotiation.cc:407] Connection header received
0809 06:30:27.050057 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:27.050064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:27.050151 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:27.050316 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:27.051329 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:27.052546 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:27.054304 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:27.054728 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:27.054898 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:27.056310 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:27.056320 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:27.056325 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:27.056342 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:27.056382 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:27.056394 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:27.056493 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:27.056733 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 06:30:27.056836 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":291,"threads_started":1}
I20260809 06:30:27.058111 6067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:27.047148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:33129 (local address 127.8.131.1:39247)
0809 06:30:27.048074 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:27.048173 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:27.049797 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:27.050532 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:27.050539 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:27.050621 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:27.051153 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:27.051169 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:27.052738 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:27.052746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:27.054149 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:27.054161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:27.054998 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:27.055007 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:27.055236 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:27.055998 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:30:27.056001 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:27.056038 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:30:27.056109 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:27.056494 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:27.056507 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:27.056517 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 06:30:27.056705 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 06:30:27.057083 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":283,"threads_started":1}
I20260809 06:30:27.059621 6061 heartbeater.cc:347] Connected to a master server at 127.8.131.62:33129
I20260809 06:30:27.059971 6061 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:27.060860 6061 heartbeater.cc:511] Master 127.8.131.62:33129 requested a full tablet report, sending...
I20260809 06:30:27.063217 5783 ts_manager.cc:194] Registered new tserver with Master: a0064f94232c4d8d8a6e0da532e793a3 (127.8.131.1:41525)
I20260809 06:30:27.065351 5783 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.1:39247
W20260809 06:30:27.390055 6069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:27.390352 6069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:27.390496 6069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:27.397892 6069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:27.397991 6069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:27.398190 6069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.2
I20260809 06:30:27.407207 6069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.131.2
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.2
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:27.408605 6069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:27.410404 6069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:27.423851 6083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:27.423851 6080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:27.423844 6081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:27.424865 6069 server_base.cc:1034] running on GCE node
I20260809 06:30:27.425616 6069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:27.426743 6069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:27.427987 6069 hybrid_clock.cc:648] HybridClock initialized: now 1786257027427928 us; error 46 us; skew 500 ppm
I20260809 06:30:27.428346 6069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:27.430974 6069 webserver.cc:460] Webserver started at http://127.8.131.2:33089/ using document root <none> and password file <none>
I20260809 06:30:27.431766 6069 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:27.431913 6069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:27.432277 6069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:27.435286 6069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/data/instance:
uuid: "9cf92fea4fe54cc492cdef6c6b688a50"
format_stamp: "Formatted at 2026-08-09 06:30:27 on dist-test-slave-mjjr"
server_key: "866cad8d237e0988dca4dc2d2600ebd5"
server_key_iv: "c39854ff07af544c656403857ffb2485"
server_key_version: "encryptionkey@0"
I20260809 06:30:27.436160 6069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance:
uuid: "9cf92fea4fe54cc492cdef6c6b688a50"
format_stamp: "Formatted at 2026-08-09 06:30:27 on dist-test-slave-mjjr"
server_key: "866cad8d237e0988dca4dc2d2600ebd5"
server_key_iv: "c39854ff07af544c656403857ffb2485"
server_key_version: "encryptionkey@0"
I20260809 06:30:27.441418 6069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:30:27.444909 6089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:27.446323 6069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:30:27.446545 6069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/wal
uuid: "9cf92fea4fe54cc492cdef6c6b688a50"
format_stamp: "Formatted at 2026-08-09 06:30:27 on dist-test-slave-mjjr"
server_key: "866cad8d237e0988dca4dc2d2600ebd5"
server_key_iv: "c39854ff07af544c656403857ffb2485"
server_key_version: "encryptionkey@0"
I20260809 06:30:27.446805 6069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:27.466156 6069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:27.467021 6069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:27.467335 6069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:27.468436 6069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:27.469964 6069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:27.470101 6069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:27.470250 6069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:27.470350 6069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:27.508533 6069 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.2:36941
I20260809 06:30:27.508545 6250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.2:36941 every 8 connection(s)
I20260809 06:30:27.510253 6069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/data/info.pb
I20260809 06:30:27.517794 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 6069
I20260809 06:30:27.517975 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/wal/instance
I20260809 06:30:27.518432 8716 external_mini_cluster.cc:1428] Setting key ac4687a7095423a2f68ef6070c2ac1ff
I20260809 06:30:27.521524 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.131.3:0
--local_ip_for_outbound_sockets=127.8.131.3
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33129
--builtin_ntp_servers=127.8.131.20:38835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 06:30:27.523307 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:27.512979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.2:56583 (local address 127.8.131.62:33129)
0809 06:30:27.513194 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:27.513200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:27.514644 (+ 1444us) server_negotiation.cc:407] Connection header received
0809 06:30:27.516434 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:27.516442 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:27.516522 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:27.516631 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:27.517677 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:27.518639 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:27.520578 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:27.520958 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:27.521058 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:27.522600 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:27.522605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:27.522608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:27.522625 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:27.522655 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:27.522666 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:27.522738 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:27.523006 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 06:30:27.523078 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:30:27.524724 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:27.513440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:33129 (local address 127.8.131.2:56583)
0809 06:30:27.514470 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:27.514515 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:27.516170 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:27.516823 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:27.516830 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:27.516910 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:27.517490 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:27.517515 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:27.518785 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:27.518790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:27.520415 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:27.520427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:27.521179 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:27.521188 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:27.521499 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:27.522254 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:30:27.522260 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:27.522310 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:30:27.522405 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:27.522763 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:27.522776 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:27.522787 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 06:30:27.522981 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 06:30:27.523386 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":364,"threads_started":1}
I20260809 06:30:27.526125 6251 heartbeater.cc:347] Connected to a master server at 127.8.131.62:33129
I20260809 06:30:27.526427 6251 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:27.527415 6251 heartbeater.cc:511] Master 127.8.131.62:33129 requested a full tablet report, sending...
I20260809 06:30:27.528885 5783 ts_manager.cc:194] Registered new tserver with Master: 9cf92fea4fe54cc492cdef6c6b688a50 (127.8.131.2:36941)
I20260809 06:30:27.529763 5783 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.2:56583
W20260809 06:30:27.870857 6255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:27.871317 6255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:27.871495 6255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:27.880038 6255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:27.880177 6255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:27.880355 6255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.131.3
I20260809 06:30:27.889732 6255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:38835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.131.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.131.3
--webserver_port=0
--tserver_master_addrs=127.8.131.62:33129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.131.3
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:27.891110 6255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:27.892980 6255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:27.906453 6261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:27.906410 6265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:27.906489 6262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:27.907351 6255 server_base.cc:1034] running on GCE node
I20260809 06:30:27.908015 6255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:27.908986 6255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:27.910228 6255 hybrid_clock.cc:648] HybridClock initialized: now 1786257027910190 us; error 50 us; skew 500 ppm
I20260809 06:30:27.910557 6255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 06:30:27.912860 6255 webserver.cc:460] Webserver started at http://127.8.131.3:39637/ using document root <none> and password file <none>
I20260809 06:30:27.913563 6255 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:27.913697 6255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:27.914026 6255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:27.916918 6255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/data/instance:
uuid: "993b25ace6e04bf0b886c08ec946c790"
format_stamp: "Formatted at 2026-08-09 06:30:27 on dist-test-slave-mjjr"
server_key: "0640c18fe3e3b54870ed0b53dbe27153"
server_key_iv: "093767586a04f04b373880a298431277"
server_key_version: "encryptionkey@0"
I20260809 06:30:27.917747 6255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance:
uuid: "993b25ace6e04bf0b886c08ec946c790"
format_stamp: "Formatted at 2026-08-09 06:30:27 on dist-test-slave-mjjr"
server_key: "0640c18fe3e3b54870ed0b53dbe27153"
server_key_iv: "093767586a04f04b373880a298431277"
server_key_version: "encryptionkey@0"
I20260809 06:30:27.922817 6255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:30:27.926277 6274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:27.927754 6255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:27.927958 6255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/wal
uuid: "993b25ace6e04bf0b886c08ec946c790"
format_stamp: "Formatted at 2026-08-09 06:30:27 on dist-test-slave-mjjr"
server_key: "0640c18fe3e3b54870ed0b53dbe27153"
server_key_iv: "093767586a04f04b373880a298431277"
server_key_version: "encryptionkey@0"
I20260809 06:30:27.928186 6255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:27.954330 6255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:27.955201 6255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:27.955516 6255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:27.956558 6255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 06:30:27.958040 6255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 06:30:27.958166 6255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:27.958313 6255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 06:30:27.958410 6255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:27.992846 6255 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.3:46719
I20260809 06:30:27.992935 6434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.3:46719 every 8 connection(s)
I20260809 06:30:27.994222 6255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/data/info.pb
I20260809 06:30:27.994650 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 6255
I20260809 06:30:27.994834 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/wal/instance
I20260809 06:30:27.995487 8716 external_mini_cluster.cc:1428] Setting key 2c6aeba5c9c99f625ac72179f1c85b79
I20260809 06:30:28.007201 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:27.997114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:43263 (local address 127.8.131.62:33129)
0809 06:30:27.997331 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:27.997337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:27.998995 (+ 1658us) server_negotiation.cc:407] Connection header received
0809 06:30:28.000952 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.000956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.001024 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.001109 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:28.002128 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.002929 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.004525 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.004945 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.005055 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.006375 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:28.006380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:28.006383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:28.006399 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:28.006429 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:28.006440 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:28.006563 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.006865 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.006943 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:30:28.008163 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:27.997754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:33129 (local address 127.8.131.3:43263)
0809 06:30:27.998850 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:27.998891 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:28.000710 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:28.001314 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:28.001321 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:28.001410 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:30:28.001959 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.001975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.003086 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.003090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.004375 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.004386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.005155 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.005161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.005372 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.006087 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 06:30:28.006090 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:30:28.006126 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 06:30:28.006201 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:30:28.006631 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:30:28.006640 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:30:28.006647 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 06:30:28.006806 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 06:30:28.007077 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":324,"threads_started":1}
I20260809 06:30:28.009630 6436 heartbeater.cc:347] Connected to a master server at 127.8.131.62:33129
I20260809 06:30:28.009953 6436 heartbeater.cc:464] Registering TS with master...
I20260809 06:30:28.010805 6436 heartbeater.cc:511] Master 127.8.131.62:33129 requested a full tablet report, sending...
I20260809 06:30:28.012171 5783 ts_manager.cc:194] Registered new tserver with Master: 993b25ace6e04bf0b886c08ec946c790 (127.8.131.3:46719)
I20260809 06:30:28.012965 5783 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.131.3:43263
I20260809 06:30:28.023314 8716 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 06:30:28.034361 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.027972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.8.131.62:33129)
0809 06:30:28.028218 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.028227 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.028642 (+ 415us) server_negotiation.cc:407] Connection header received
0809 06:30:28.028791 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.028794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.028867 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.028950 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:28.029568 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.030313 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.031878 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.032388 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.032494 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.033598 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:28.033603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:28.033606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:28.033623 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:28.033655 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:28.033667 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:28.033739 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.033918 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.034055 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:30:28.038355 5783 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 06:30:28.045385 5783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 06:30:28.069130 6061 heartbeater.cc:503] Master 127.8.131.62:33129 was elected leader, sending a full tablet report...
I20260809 06:30:28.080957 6459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.058470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.8.131.3:46719)
0809 06:30:28.072516 (+ 14046us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.072532 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.072860 (+ 328us) server_negotiation.cc:407] Connection header received
0809 06:30:28.073195 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.073204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.073468 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.073684 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:28.074923 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.076541 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.078533 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.079722 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.079875 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.080191 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.080413 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.080580 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13565,"thread_start_us":13812,"threads_started":1}
I20260809 06:30:28.080999 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.058092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.3:46719 (local address 127.0.0.1:33616)
0809 06:30:28.072728 (+ 14636us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:28.072749 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:28.072973 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:28.073977 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:28.073983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:28.074015 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:28.074731 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.074744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.076655 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.076662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.078279 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.078292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.079918 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.079925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.080125 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.080186 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 06:30:28.080363 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 06:30:28.080560 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14472,"mutex_wait_us":30,"thread_start_us":11565,"threads_started":1}
I20260809 06:30:28.082798 6457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.057887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.8.131.2:36941)
0809 06:30:28.072968 (+ 15081us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.072976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.073003 (+ 27us) server_negotiation.cc:407] Connection header received
0809 06:30:28.073108 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.073116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.073340 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.074324 (+ 984us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:28.074584 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.075937 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.079060 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.081659 (+ 2599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.081804 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.082040 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.082256 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.082427 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14914,"thread_start_us":14780,"threads_started":1}
I20260809 06:30:28.083303 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.073879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41525 (local address 127.0.0.1:46730)
0809 06:30:28.074619 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:28.074638 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:28.074795 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:28.075829 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:28.075833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:28.075858 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:28.076361 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.076373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.078420 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.078428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.080393 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.080408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.082613 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.082619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.082824 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.082864 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 06:30:28.082956 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 06:30:28.083084 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":454,"threads_started":1}
I20260809 06:30:28.084012 6458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.057557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:36941 (local address 127.0.0.1:49004)
0809 06:30:28.069123 (+ 11566us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:28.069181 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:28.069426 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:28.073622 (+ 4196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:28.073631 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:28.073664 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:28.074172 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.074191 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.076109 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.076116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.078843 (+ 2727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.078860 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.081885 (+ 3025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.081891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.082060 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.082091 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 06:30:28.082191 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 06:30:28.082322 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11199,"mutex_wait_us":76,"thread_start_us":14845,"threads_started":1}
I20260809 06:30:28.084146 6461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.073215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46730 (local address 127.8.131.1:41525)
0809 06:30:28.073885 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.073892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.074758 (+ 866us) server_negotiation.cc:407] Connection header received
0809 06:30:28.075017 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.075026 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.075289 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.075485 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:28.076584 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.078246 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.080633 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.082087 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.082245 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.082491 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.083548 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.083759 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":331,"threads_started":1}
I20260809 06:30:28.085350 6348 tablet_service.cc:1467] Processing CreateTablet for tablet 970ea6013f774b3181f631004fa3fbff (DEFAULT_TABLE table=test-table [id=3e802abda04a48649eba9f932e35929b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:28.087010 6348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 970ea6013f774b3181f631004fa3fbff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:28.087400 5969 tablet_service.cc:1467] Processing CreateTablet for tablet 970ea6013f774b3181f631004fa3fbff (DEFAULT_TABLE table=test-table [id=3e802abda04a48649eba9f932e35929b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:28.087517 6156 tablet_service.cc:1467] Processing CreateTablet for tablet 970ea6013f774b3181f631004fa3fbff (DEFAULT_TABLE table=test-table [id=3e802abda04a48649eba9f932e35929b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 06:30:28.088670 5969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 970ea6013f774b3181f631004fa3fbff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:28.088841 6156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 970ea6013f774b3181f631004fa3fbff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:28.099789 6463 tablet_bootstrap.cc:492] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: Bootstrap starting.
I20260809 06:30:28.102679 6464 tablet_bootstrap.cc:492] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: Bootstrap starting.
I20260809 06:30:28.103519 6465 tablet_bootstrap.cc:492] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: Bootstrap starting.
I20260809 06:30:28.104854 6463 tablet_bootstrap.cc:654] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:28.106197 6463 log.cc:824] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:28.107304 6465 tablet_bootstrap.cc:654] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:28.108615 6465 log.cc:824] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:28.109309 6463 tablet_bootstrap.cc:492] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: No bootstrap required, opened a new log
I20260809 06:30:28.109643 6463 ts_tablet_manager.cc:1397] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 06:30:28.110489 6464 tablet_bootstrap.cc:654] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:28.111689 6464 log.cc:824] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:28.111841 6465 tablet_bootstrap.cc:492] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: No bootstrap required, opened a new log
I20260809 06:30:28.112104 6465 ts_tablet_manager.cc:1397] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260809 06:30:28.115306 6464 tablet_bootstrap.cc:492] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: No bootstrap required, opened a new log
I20260809 06:30:28.115573 6464 ts_tablet_manager.cc:1397] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20260809 06:30:28.117803 6463 raft_consensus.cc:348] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } }
I20260809 06:30:28.118185 6463 raft_consensus.cc:374] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:28.118290 6463 raft_consensus.cc:729] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0064f94232c4d8d8a6e0da532e793a3, State: Initialized, Role: FOLLOWER
I20260809 06:30:28.118929 6463 consensus_queue.cc:260] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } }
I20260809 06:30:28.120733 6463 ts_tablet_manager.cc:1428] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260809 06:30:28.122643 6465 raft_consensus.cc:348] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } }
I20260809 06:30:28.123083 6465 raft_consensus.cc:374] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:28.123178 6465 raft_consensus.cc:729] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993b25ace6e04bf0b886c08ec946c790, State: Initialized, Role: FOLLOWER
I20260809 06:30:28.123811 6465 consensus_queue.cc:260] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } }
I20260809 06:30:28.125209 6436 heartbeater.cc:503] Master 127.8.131.62:33129 was elected leader, sending a full tablet report...
I20260809 06:30:28.125756 6465 ts_tablet_manager.cc:1428] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260809 06:30:28.127135 6464 raft_consensus.cc:348] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } }
I20260809 06:30:28.127578 6464 raft_consensus.cc:374] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:28.127714 6464 raft_consensus.cc:729] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cf92fea4fe54cc492cdef6c6b688a50, State: Initialized, Role: FOLLOWER
I20260809 06:30:28.128350 6464 consensus_queue.cc:260] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } }
I20260809 06:30:28.129554 6251 heartbeater.cc:503] Master 127.8.131.62:33129 was elected leader, sending a full tablet report...
I20260809 06:30:28.130069 6464 ts_tablet_manager.cc:1428] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: Time spent starting tablet: real 0.014s user 0.003s sys 0.009s
I20260809 06:30:28.131937 6471 raft_consensus.cc:482] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 06:30:28.132200 6471 raft_consensus.cc:504] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } }
I20260809 06:30:28.133921 6471 leader_election.cc:288] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cf92fea4fe54cc492cdef6c6b688a50 (127.8.131.2:36941), a0064f94232c4d8d8a6e0da532e793a3 (127.8.131.1:41525)
I20260809 06:30:28.139817 6439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.134169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.2:36941 (local address 127.8.131.3:50985)
0809 06:30:28.134355 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:28.134380 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:28.134497 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:28.135124 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:28.135129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:28.135154 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:28.135368 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.135381 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.136642 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.136649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.137983 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.137996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.139021 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.139026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.139220 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.139244 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 06:30:28.139357 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 06:30:28.139498 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 06:30:28.139818 6457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.134261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:50985 (local address 127.8.131.2:36941)
0809 06:30:28.134467 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.134474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.134496 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:30:28.134675 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.134678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.134777 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.134883 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:28.135536 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.136513 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.138139 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.138871 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.139012 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.139100 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.139398 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.139521 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 06:30:28.140350 6461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.134903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.131.3:43623 (local address 127.8.131.1:41525)
0809 06:30:28.135092 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.135096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.135264 (+ 168us) server_negotiation.cc:407] Connection header received
0809 06:30:28.135438 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.135442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.135505 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.135580 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 06:30:28.136189 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.136872 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.138629 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.139396 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.139533 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.139632 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.140033 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.140151 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 06:30:28.140862 6474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.134561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.1:41525 (local address 127.8.131.3:43623)
0809 06:30:28.135181 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0809 06:30:28.135207 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 06:30:28.135336 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:30:28.135623 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:30:28.135628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:30:28.135681 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 06:30:28.136043 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.136054 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.137032 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.137037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.138445 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:30:28.138458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.139584 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.139590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:30:28.139815 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.139839 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 06:30:28.140547 (+ 708us) client_negotiation.cc:240] Negotiation successful
0809 06:30:28.140654 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":298,"threads_started":1}
I20260809 06:30:28.140762 6187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970ea6013f774b3181f631004fa3fbff" candidate_uuid: "993b25ace6e04bf0b886c08ec946c790" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" is_pre_election: true
I20260809 06:30:28.141255 6187 raft_consensus.cc:2454] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 993b25ace6e04bf0b886c08ec946c790 in term 0.
I20260809 06:30:28.141916 6280 leader_election.cc:302] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 993b25ace6e04bf0b886c08ec946c790, 9cf92fea4fe54cc492cdef6c6b688a50; no voters:
I20260809 06:30:28.141911 5998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970ea6013f774b3181f631004fa3fbff" candidate_uuid: "993b25ace6e04bf0b886c08ec946c790" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0064f94232c4d8d8a6e0da532e793a3" is_pre_election: true
I20260809 06:30:28.142385 6471 raft_consensus.cc:2781] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 06:30:28.142427 5998 raft_consensus.cc:2454] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 993b25ace6e04bf0b886c08ec946c790 in term 0.
I20260809 06:30:28.142542 6471 raft_consensus.cc:482] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 06:30:28.142619 6471 raft_consensus.cc:3037] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:28.145153 6471 raft_consensus.cc:504] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } }
I20260809 06:30:28.145709 6471 leader_election.cc:288] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [CANDIDATE]: Term 1 election: Requested vote from peers 9cf92fea4fe54cc492cdef6c6b688a50 (127.8.131.2:36941), a0064f94232c4d8d8a6e0da532e793a3 (127.8.131.1:41525)
I20260809 06:30:28.146132 6187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970ea6013f774b3181f631004fa3fbff" candidate_uuid: "993b25ace6e04bf0b886c08ec946c790" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cf92fea4fe54cc492cdef6c6b688a50"
I20260809 06:30:28.146214 5998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970ea6013f774b3181f631004fa3fbff" candidate_uuid: "993b25ace6e04bf0b886c08ec946c790" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0064f94232c4d8d8a6e0da532e793a3"
I20260809 06:30:28.146345 6187 raft_consensus.cc:3037] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:28.146453 5998 raft_consensus.cc:3037] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:28.148882 5998 raft_consensus.cc:2454] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 993b25ace6e04bf0b886c08ec946c790 in term 1.
I20260809 06:30:28.149011 6187 raft_consensus.cc:2454] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 993b25ace6e04bf0b886c08ec946c790 in term 1.
I20260809 06:30:28.149295 6280 leader_election.cc:302] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 993b25ace6e04bf0b886c08ec946c790, a0064f94232c4d8d8a6e0da532e793a3; no voters:
I20260809 06:30:28.149590 6471 raft_consensus.cc:2781] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:28.150051 6471 raft_consensus.cc:686] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 1 LEADER]: Becoming Leader. State: Replica: 993b25ace6e04bf0b886c08ec946c790, State: Running, Role: LEADER
I20260809 06:30:28.150576 6471 consensus_queue.cc:237] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } }
I20260809 06:30:28.155577 5783 catalog_manager.cc:5495] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 reported cstate change: term changed from 0 to 1, leader changed from <none> to 993b25ace6e04bf0b886c08ec946c790 (127.8.131.3). New cstate: current_term: 1 leader_uuid: "993b25ace6e04bf0b886c08ec946c790" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "993b25ace6e04bf0b886c08ec946c790" member_type: VOTER last_known_addr { host: "127.8.131.3" port: 46719 } health_report { overall_health: HEALTHY } } }
I20260809 06:30:28.174520 6459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.169365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.8.131.3:46719)
0809 06:30:28.169559 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.169566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.169715 (+ 149us) server_negotiation.cc:407] Connection header received
0809 06:30:28.169891 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.169895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.169988 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.170108 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:30:28.170458 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.171162 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.172529 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.172928 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.173021 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.173435 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:30:28.173945 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:30:28.174052 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.174152 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.174212 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:30:28.182595 5998 raft_consensus.cc:1261] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [term 1 FOLLOWER]: Refusing update from remote peer 993b25ace6e04bf0b886c08ec946c790: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:28.182595 6187 raft_consensus.cc:1261] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [term 1 FOLLOWER]: Refusing update from remote peer 993b25ace6e04bf0b886c08ec946c790: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:30:28.183588 6476 consensus_queue.cc:1035] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0064f94232c4d8d8a6e0da532e793a3" member_type: VOTER last_known_addr { host: "127.8.131.1" port: 41525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:28.183986 6471 consensus_queue.cc:1035] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cf92fea4fe54cc492cdef6c6b688a50" member_type: VOTER last_known_addr { host: "127.8.131.2" port: 36941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:30:28.210263 5783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:28.210546 5783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 06:30:28.214383 5783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc69583d081a41e58850e812415ce8b7: Sending DeleteTablet for 3 replicas of tablet 970ea6013f774b3181f631004fa3fbff
I20260809 06:30:28.215848 5969 tablet_service.cc:1514] Processing DeleteTablet for tablet 970ea6013f774b3181f631004fa3fbff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:28 UTC) from {username='slave'} at 127.0.0.1:46730
I20260809 06:30:28.215852 6156 tablet_service.cc:1514] Processing DeleteTablet for tablet 970ea6013f774b3181f631004fa3fbff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:28 UTC) from {username='slave'} at 127.0.0.1:49004
I20260809 06:30:28.216706 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 5871
I20260809 06:30:28.219413 6348 tablet_service.cc:1514] Processing DeleteTablet for tablet 970ea6013f774b3181f631004fa3fbff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 06:30:28 UTC) from {username='slave'} at 127.0.0.1:33616
I20260809 06:30:28.226071 6498 tablet_replica.cc:330] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: stopping tablet replica
I20260809 06:30:28.227221 6498 raft_consensus.cc:2229] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:28.228375 6498 raft_consensus.cc:2258] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:28.232687 6498 ts_tablet_manager.cc:1905] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:28.233564 6504 tablet_replica.cc:330] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: stopping tablet replica
I20260809 06:30:28.235528 6504 raft_consensus.cc:2229] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 1 LEADER]: Raft consensus shutting down.
I20260809 06:30:28.237954 6504 raft_consensus.cc:2258] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:28.240474 6498 ts_tablet_manager.cc:1918] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:28.241161 6494 tablet_replica.cc:330] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: stopping tablet replica
I20260809 06:30:28.241570 6498 log.cc:1196] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-0/wal/wals/970ea6013f774b3181f631004fa3fbff
I20260809 06:30:28.242257 6498 ts_tablet_manager.cc:1939] T 970ea6013f774b3181f631004fa3fbff P a0064f94232c4d8d8a6e0da532e793a3: Deleting consensus metadata
I20260809 06:30:28.242218 6494 raft_consensus.cc:2229] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 06:30:28.244103 6494 raft_consensus.cc:2258] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 06:30:28.244553 5763 catalog_manager.cc:4841] TS a0064f94232c4d8d8a6e0da532e793a3 (127.8.131.1:41525): tablet 970ea6013f774b3181f631004fa3fbff (table test-table [id=3e802abda04a48649eba9f932e35929b]) successfully deleted
I20260809 06:30:28.244676 6504 ts_tablet_manager.cc:1905] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:28.246860 6461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.218064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.8.131.1:41525)
0809 06:30:28.218333 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.218343 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.234436 (+ 16093us) server_negotiation.cc:407] Connection header received
0809 06:30:28.235814 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.235824 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.235987 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.236096 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:28.238341 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.240001 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.242303 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.243005 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.243116 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.244858 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:28.244869 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:28.244886 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:28.245762 (+ 876us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:28.245828 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:28.245856 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:28.245957 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.246265 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.246411 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 06:30:28.247025 6494 ts_tablet_manager.cc:1905] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 06:30:28.248359 6056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:28.255121 6494 ts_tablet_manager.cc:1918] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:28.255388 6494 log.cc:1196] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-1/wal/wals/970ea6013f774b3181f631004fa3fbff
I20260809 06:30:28.256134 6494 ts_tablet_manager.cc:1939] T 970ea6013f774b3181f631004fa3fbff P 9cf92fea4fe54cc492cdef6c6b688a50: Deleting consensus metadata
I20260809 06:30:28.256457 6504 ts_tablet_manager.cc:1918] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 06:30:28.256716 6504 log.cc:1196] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: Deleting WAL directory at /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786256699676692-8716-0/minicluster-data/ts-2/wal/wals/970ea6013f774b3181f631004fa3fbff
I20260809 06:30:28.257362 6504 ts_tablet_manager.cc:1939] T 970ea6013f774b3181f631004fa3fbff P 993b25ace6e04bf0b886c08ec946c790: Deleting consensus metadata
I20260809 06:30:28.258368 5763 catalog_manager.cc:4841] TS 9cf92fea4fe54cc492cdef6c6b688a50 (127.8.131.2:36941): tablet 970ea6013f774b3181f631004fa3fbff (table test-table [id=3e802abda04a48649eba9f932e35929b]) successfully deleted
I20260809 06:30:28.259166 5764 catalog_manager.cc:4841] TS 993b25ace6e04bf0b886c08ec946c790 (127.8.131.3:46719): tablet 970ea6013f774b3181f631004fa3fbff (table test-table [id=3e802abda04a48649eba9f932e35929b]) successfully deleted
I20260809 06:30:28.340632 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 5871
I20260809 06:30:28.373839 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 6069
I20260809 06:30:28.381175 6457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.374770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.8.131.2:36941)
0809 06:30:28.375041 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.375047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.375068 (+ 21us) server_negotiation.cc:407] Connection header received
0809 06:30:28.375255 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.375259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.375346 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.375457 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:28.375966 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.376724 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.378048 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.378453 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.378608 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.379774 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:28.379787 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:28.379801 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:28.380388 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:28.380440 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:28.380458 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:28.380594 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.380727 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.380837 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 06:30:28.381956 6246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:28.471258 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 6069
I20260809 06:30:28.499305 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 6255
I20260809 06:30:28.506887 6459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:28.500328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33632 (local address 127.8.131.3:46719)
0809 06:30:28.500536 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:28.500544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:28.500567 (+ 23us) server_negotiation.cc:407] Connection header received
0809 06:30:28.500631 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:28.500637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:28.500743 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:28.500881 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:28.501399 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.502262 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.503592 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:28.504153 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:28.504290 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:28.505415 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:28.505424 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:28.505439 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:28.506064 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:28.506110 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:28.506132 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:28.506265 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:28.506414 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 06:30:28.506526 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:30:28.507596 6427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:28.629027 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 6255
I20260809 06:30:28.655853 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 5737
I20260809 06:30:28.657076 5822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:28.769176 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 5737
2026-08-09T06:30:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2824 ms)
[----------] 6 tests from EncryptionPolicyTest (22924 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:28 dist-test-slave-mjjr krb5kdc[6516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:28 dist-test-slave-mjjr krb5kdc[6516](info): set up 2 sockets
Aug 09 06:30:28 dist-test-slave-mjjr krb5kdc[6516](info): commencing operation
krb5kdc: starting...
W20260809 06:30:30.105698 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:30 dist-test-slave-mjjr krb5kdc[6516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:30.402905 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:30.744303 6538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:30.744642 6538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:30.744808 6538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:30.752039 6538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:30.752190 6538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:30.752290 6538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:30.752406 6538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:30.752490 6538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:30.762279 6538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:30.763595 6538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:30.764937 6538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:30.777221 6543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:30.777184 6547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:30.777277 6545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:30.778266 6538 server_base.cc:1034] running on GCE node
I20260809 06:30:30.779121 6538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:30.780638 6538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:30.781867 6538 hybrid_clock.cc:648] HybridClock initialized: now 1786257030781813 us; error 33 us; skew 500 ppm
I20260809 06:30:30.782174 6538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:30 dist-test-slave-mjjr krb5kdc[6516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:30.785416 6538 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:30.786872 6538 webserver.cc:460] Webserver started at http://127.0.0.1:45683/ using document root <none> and password file <none>
I20260809 06:30:30.787520 6538 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:30.787621 6538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:30.787966 6538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:30.790943 6538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "698234d95313461bb1f154d23ae7c3ec"
format_stamp: "Formatted at 2026-08-09 06:30:30 on dist-test-slave-mjjr"
server_key: "e0ace20534d104c29eebb7be0335a6da"
server_key_iv: "629e914442a8ecca560f736e555b2988"
server_key_version: "encryptionkey@0"
I20260809 06:30:30.791703 6538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "698234d95313461bb1f154d23ae7c3ec"
format_stamp: "Formatted at 2026-08-09 06:30:30 on dist-test-slave-mjjr"
server_key: "e0ace20534d104c29eebb7be0335a6da"
server_key_iv: "629e914442a8ecca560f736e555b2988"
server_key_version: "encryptionkey@0"
I20260809 06:30:30.796777 6538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 06:30:30.800257 6557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:30.801553 6538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:30:30.801694 6538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "698234d95313461bb1f154d23ae7c3ec"
format_stamp: "Formatted at 2026-08-09 06:30:30 on dist-test-slave-mjjr"
server_key: "e0ace20534d104c29eebb7be0335a6da"
server_key_iv: "629e914442a8ecca560f736e555b2988"
server_key_version: "encryptionkey@0"
I20260809 06:30:30.801859 6538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:30.829468 6538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:30.833581 6538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:30.833832 6538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:30.853207 6538 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44889
I20260809 06:30:30.853247 6634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44889 every 8 connection(s)
I20260809 06:30:30.854560 6538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:30.858238 6635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:30.862243 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 6538
I20260809 06:30:30.862469 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:30.863138 8716 external_mini_cluster.cc:1428] Setting key ca86c82f1efb2ee8b4c19d94291f8cf0
I20260809 06:30:30.869208 6635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec: Bootstrap starting.
I20260809 06:30:30.873148 6635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:30.874317 6635 log.cc:824] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:30.877876 6635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec: No bootstrap required, opened a new log
Aug 09 06:30:30 dist-test-slave-mjjr krb5kdc[6516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 06:30:30.885692 6635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698234d95313461bb1f154d23ae7c3ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44889 } }
I20260809 06:30:30.886070 6635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:30.886219 6635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 698234d95313461bb1f154d23ae7c3ec, State: Initialized, Role: FOLLOWER
I20260809 06:30:30.886847 6635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698234d95313461bb1f154d23ae7c3ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44889 } }
I20260809 06:30:30.887107 6635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:30.887281 6635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:30.887471 6635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:30.887733 6641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:30.865217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58815 (local address 127.0.0.1:44889)
0809 06:30:30.866068 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:30.866079 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:30.866121 (+ 42us) server_negotiation.cc:407] Connection header received
0809 06:30:30.867902 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:30.867924 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:30.868276 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:30.868574 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:30.870459 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:30.871976 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:30.873835 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:30.874595 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:30.874737 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:30.879820 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:30.879828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:30.879842 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:30.879880 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:30.883137 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:30.884016 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:30.884025 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:30.884031 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:30.884214 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:30.884710 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:30.884714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:30.884720 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:30.885212 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:30.885543 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:30.886047 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 06:30:30.886407 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":419,"threads_started":1}
I20260809 06:30:30.890483 6635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698234d95313461bb1f154d23ae7c3ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44889 } }
I20260809 06:30:30.891034 6635 leader_election.cc:302] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 698234d95313461bb1f154d23ae7c3ec; no voters:
I20260809 06:30:30.891713 6635 leader_election.cc:288] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:30.891847 6649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:30.891938 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:30.894199 6635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:30.897010 6649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [term 1 LEADER]: Becoming Leader. State: Replica: 698234d95313461bb1f154d23ae7c3ec, State: Running, Role: LEADER
I20260809 06:30:30.898263 6649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698234d95313461bb1f154d23ae7c3ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44889 } }
I20260809 06:30:30.901723 6650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "698234d95313461bb1f154d23ae7c3ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698234d95313461bb1f154d23ae7c3ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44889 } } }
I20260809 06:30:30.902249 6650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:30.902978 6653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 698234d95313461bb1f154d23ae7c3ec. Latest consensus state: current_term: 1 leader_uuid: "698234d95313461bb1f154d23ae7c3ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "698234d95313461bb1f154d23ae7c3ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44889 } } }
I20260809 06:30:30.903313 6653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:30.909977 6671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:30.915062 6671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:30.919275 6641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:30.897438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59069 (local address 127.0.0.1:44889)
0809 06:30:30.897662 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:30.897672 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:30.899112 (+ 1440us) server_negotiation.cc:407] Connection header received
0809 06:30:30.899315 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:30.899323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:30.899422 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:30.899541 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:30.901714 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:30.903101 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:30.906094 (+ 2993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:30.907067 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:30.907191 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:30.911050 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:30.911060 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:30.911068 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:30.911119 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:30.915116 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:30.916113 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:30.916119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:30.916122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:30.916292 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:30.916889 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:30.916894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:30.916897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:30.917382 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:30.918545 (+ 1163us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:30.918735 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 06:30:30.918924 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 06:30:30.924646 6671 catalog_manager.cc:1347] Generated new cluster ID: 9750ed1a1b7b4904a53b2dd135a425bb
I20260809 06:30:30.924762 6671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:30.946798 6671 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:30.948241 6671 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:30.955741 6671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 698234d95313461bb1f154d23ae7c3ec: Generated new TSK 0
I20260809 06:30:30.956569 6671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:30.963800 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 6538
I20260809 06:30:30.965493 6629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:31.058043 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 6538
W20260809 06:30:31.081024 6513 connection.cc:535] client connection to 127.0.0.1:44889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44889): Connection reset by peer (error 104)
2026-08-09T06:30:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:31 dist-test-slave-mjjr krb5kdc[6691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:31 dist-test-slave-mjjr krb5kdc[6691](info): set up 2 sockets
Aug 09 06:30:31 dist-test-slave-mjjr krb5kdc[6691](info): commencing operation
krb5kdc: starting...
W20260809 06:30:32.390995 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:32 dist-test-slave-mjjr krb5kdc[6691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:32.703375 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40175
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:33.048427 6708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:33.048722 6708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:33.048940 6708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:33.056231 6708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:33.056339 6708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:33.056444 6708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:33.056555 6708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:33.056636 6708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:33.066546 6708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40175
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:33.067942 6708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:33.069289 6708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:33.081660 6718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:33.081813 6717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:33.081771 6720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:33.082684 6708 server_base.cc:1034] running on GCE node
I20260809 06:30:33.083424 6708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:33.084874 6708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:33.086135 6708 hybrid_clock.cc:648] HybridClock initialized: now 1786257033086070 us; error 55 us; skew 500 ppm
I20260809 06:30:33.086530 6708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:33 dist-test-slave-mjjr krb5kdc[6691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257033, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:33.093613 6708 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:33.095253 6708 webserver.cc:460] Webserver started at http://127.0.0.1:34475/ using document root <none> and password file <none>
I20260809 06:30:33.096061 6708 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:33.096220 6708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:33.096540 6708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:33.099737 6708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "90bfce7e2ea341de91553e5d9f3227ca"
format_stamp: "Formatted at 2026-08-09 06:30:33 on dist-test-slave-mjjr"
server_key: "3ebb5631cc073687a2146f3a746a8756"
server_key_iv: "abb6c83cacaa279f739ffb10d34e5532"
server_key_version: "encryptionkey@0"
I20260809 06:30:33.100557 6708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "90bfce7e2ea341de91553e5d9f3227ca"
format_stamp: "Formatted at 2026-08-09 06:30:33 on dist-test-slave-mjjr"
server_key: "3ebb5631cc073687a2146f3a746a8756"
server_key_iv: "abb6c83cacaa279f739ffb10d34e5532"
server_key_version: "encryptionkey@0"
I20260809 06:30:33.105746 6708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 06:30:33.109316 6728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:33.110646 6708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:30:33.110853 6708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "90bfce7e2ea341de91553e5d9f3227ca"
format_stamp: "Formatted at 2026-08-09 06:30:33 on dist-test-slave-mjjr"
server_key: "3ebb5631cc073687a2146f3a746a8756"
server_key_iv: "abb6c83cacaa279f739ffb10d34e5532"
server_key_version: "encryptionkey@0"
I20260809 06:30:33.111076 6708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:33.138787 6708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:33.143143 6708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:33.143460 6708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:33.162482 6708 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40175
I20260809 06:30:33.162531 6813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40175 every 8 connection(s)
I20260809 06:30:33.163903 6708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:33.164997 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 6708
I20260809 06:30:33.165189 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:33.165766 8716 external_mini_cluster.cc:1428] Setting key 14917c1be62d1cad883e45105e40ad7c
I20260809 06:30:33.168431 6815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:33.179173 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:33.167550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44457 (local address 127.0.0.1:40175)
0809 06:30:33.168401 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:33.168411 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:33.168455 (+ 44us) server_negotiation.cc:407] Connection header received
0809 06:30:33.170095 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:33.170114 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:33.170477 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:33.170791 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:33.171362 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:33.172733 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:33.174715 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:33.175389 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:33.175519 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:33.176956 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:33.176963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:33.176975 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:33.177520 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:33.177595 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:33.177622 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:33.177745 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:33.178067 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 06:30:33.178247 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":400,"threads_started":1}
I20260809 06:30:33.180500 6815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca: Bootstrap starting.
I20260809 06:30:33.182875 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:33.183708 6815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:33.184926 6815 log.cc:824] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:33.188306 6815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca: No bootstrap required, opened a new log
I20260809 06:30:33.194865 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:33.188143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42055 (local address 127.0.0.1:40175)
0809 06:30:33.188326 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:33.188333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:33.188713 (+ 380us) server_negotiation.cc:407] Connection header received
0809 06:30:33.188866 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:33.188871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:33.188941 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:33.189053 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:33.189521 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:33.190704 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:33.192182 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:33.192866 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:33.192990 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:33.194049 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:33.194055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:33.194060 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:33.194081 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:33.194131 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:33.194155 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:33.194257 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:33.194449 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 06:30:33.194505 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 06:30:33.197722 6815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bfce7e2ea341de91553e5d9f3227ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } }
I20260809 06:30:33.198096 6815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:33.198220 6815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90bfce7e2ea341de91553e5d9f3227ca, State: Initialized, Role: FOLLOWER
I20260809 06:30:33.198794 6815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bfce7e2ea341de91553e5d9f3227ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } }
I20260809 06:30:33.199033 6815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:33.199193 6815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:33.199373 6815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:33.201676 6815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bfce7e2ea341de91553e5d9f3227ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } }
I20260809 06:30:33.202169 6815 leader_election.cc:302] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90bfce7e2ea341de91553e5d9f3227ca; no voters:
I20260809 06:30:33.202745 6815 leader_election.cc:288] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:33.202896 6831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:33.203670 6831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [term 1 LEADER]: Becoming Leader. State: Replica: 90bfce7e2ea341de91553e5d9f3227ca, State: Running, Role: LEADER
I20260809 06:30:33.204208 6831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bfce7e2ea341de91553e5d9f3227ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } }
I20260809 06:30:33.204877 6815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:33.206933 6833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90bfce7e2ea341de91553e5d9f3227ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bfce7e2ea341de91553e5d9f3227ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } } }
I20260809 06:30:33.207379 6833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:33.207379 6834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90bfce7e2ea341de91553e5d9f3227ca. Latest consensus state: current_term: 1 leader_uuid: "90bfce7e2ea341de91553e5d9f3227ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bfce7e2ea341de91553e5d9f3227ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } } }
I20260809 06:30:33.207839 6834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:33.209770 6838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:33.214701 6838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:33.229267 6838 catalog_manager.cc:1347] Generated new cluster ID: 46481f0d2ac14ece8764999f3b88bc3c
I20260809 06:30:33.229394 6838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:33.267846 6838 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:33.269181 6838 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:33.280550 6838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90bfce7e2ea341de91553e5d9f3227ca: Generated new TSK 0
I20260809 06:30:33.281349 6838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:33.297458 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 6708
W20260809 06:30:33.298784 6808 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44457
W20260809 06:30:33.299549 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 6708: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:30:33.299749 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 6708
W20260809 06:30:33.323390 6689 connection.cc:535] client connection to 127.0.0.1:40175 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40175): Connection reset by peer (error 104)
2026-08-09T06:30:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2242 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:33 dist-test-slave-mjjr krb5kdc[6862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:33 dist-test-slave-mjjr krb5kdc[6862](info): set up 2 sockets
Aug 09 06:30:33 dist-test-slave-mjjr krb5kdc[6862](info): commencing operation
krb5kdc: starting...
W20260809 06:30:34.621459 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:34 dist-test-slave-mjjr krb5kdc[6862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:34.925181 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:35.269420 6884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:35.269758 6884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:35.269932 6884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:35.277201 6884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:35.277359 6884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:35.277463 6884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:35.277567 6884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:35.277649 6884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:35.287495 6884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40177
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:35.288846 6884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:35.290248 6884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:35.302011 6892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:35.302539 6897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:35.302640 6895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:35.303463 6884 server_base.cc:1034] running on GCE node
I20260809 06:30:35.304339 6884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:35.305696 6884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:35.306943 6884 hybrid_clock.cc:648] HybridClock initialized: now 1786257035306890 us; error 37 us; skew 500 ppm
I20260809 06:30:35.307324 6884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:35 dist-test-slave-mjjr krb5kdc[6862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:35.314831 6884 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:35.316578 6884 webserver.cc:460] Webserver started at http://127.0.0.1:38613/ using document root <none> and password file <none>
I20260809 06:30:35.317291 6884 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:35.317432 6884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:35.317756 6884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:35.320840 6884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "4649fd49ab8c40b7be64e69f30a8892e"
format_stamp: "Formatted at 2026-08-09 06:30:35 on dist-test-slave-mjjr"
server_key: "a83bebc99d32363fe83094ee07fd4fe0"
server_key_iv: "5beb6146f1f78d8ff1fad50fa610deb7"
server_key_version: "encryptionkey@0"
I20260809 06:30:35.321624 6884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "4649fd49ab8c40b7be64e69f30a8892e"
format_stamp: "Formatted at 2026-08-09 06:30:35 on dist-test-slave-mjjr"
server_key: "a83bebc99d32363fe83094ee07fd4fe0"
server_key_iv: "5beb6146f1f78d8ff1fad50fa610deb7"
server_key_version: "encryptionkey@0"
I20260809 06:30:35.326897 6884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 06:30:35.330303 6907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:35.331596 6884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 06:30:35.331823 6884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "4649fd49ab8c40b7be64e69f30a8892e"
format_stamp: "Formatted at 2026-08-09 06:30:35 on dist-test-slave-mjjr"
server_key: "a83bebc99d32363fe83094ee07fd4fe0"
server_key_iv: "5beb6146f1f78d8ff1fad50fa610deb7"
server_key_version: "encryptionkey@0"
I20260809 06:30:35.332039 6884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:35.337963 6884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:35.342130 6884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:35.342425 6884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:35.362253 6884 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40177
I20260809 06:30:35.362342 6990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40177 every 8 connection(s)
I20260809 06:30:35.363716 6884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:35.365533 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 6884
I20260809 06:30:35.365741 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:35.366194 8716 external_mini_cluster.cc:1428] Setting key 8211c1e3b7181c15c21abec42dd765ca
I20260809 06:30:35.368081 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:35.375505 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:35.368061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42733 (local address 127.0.0.1:40177)
0809 06:30:35.368978 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:35.368994 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:35.369048 (+ 54us) server_negotiation.cc:407] Connection header received
0809 06:30:35.371371 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:35.371399 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:35.371893 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:35.372357 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:35.373431 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:35.373442 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:35.373460 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:35.373481 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:35.373582 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:35.373613 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:35.373846 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:35.373999 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 06:30:35.374196 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":354,"threads_started":1}
I20260809 06:30:35.379211 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:35.382649 6992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e: Bootstrap starting.
I20260809 06:30:35.386261 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:35.383751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57267 (local address 127.0.0.1:40177)
0809 06:30:35.383937 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:35.383943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:35.384276 (+ 333us) server_negotiation.cc:407] Connection header received
0809 06:30:35.384427 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:35.384435 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:35.384517 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:35.384632 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:35.385545 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:35.385551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:35.385556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:35.385577 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:35.385632 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:35.385662 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:35.385815 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:35.385875 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 06:30:35.385937 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 06:30:35.386773 6992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:35.388077 6992 log.cc:824] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:35.390894 6992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e: No bootstrap required, opened a new log
I20260809 06:30:35.399164 6992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4649fd49ab8c40b7be64e69f30a8892e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40177 } }
I20260809 06:30:35.399712 6992 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:35.399811 6992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4649fd49ab8c40b7be64e69f30a8892e, State: Initialized, Role: FOLLOWER
I20260809 06:30:35.400357 6992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4649fd49ab8c40b7be64e69f30a8892e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40177 } }
I20260809 06:30:35.400579 6992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:35.400691 6992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:35.400832 6992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:35.403182 6992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4649fd49ab8c40b7be64e69f30a8892e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40177 } }
I20260809 06:30:35.403802 6992 leader_election.cc:302] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4649fd49ab8c40b7be64e69f30a8892e; no voters:
I20260809 06:30:35.404422 6992 leader_election.cc:288] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:35.404537 7003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:35.405246 7003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [term 1 LEADER]: Becoming Leader. State: Replica: 4649fd49ab8c40b7be64e69f30a8892e, State: Running, Role: LEADER
I20260809 06:30:35.405901 7003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4649fd49ab8c40b7be64e69f30a8892e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40177 } }
I20260809 06:30:35.407017 6992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:35.409255 7005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4649fd49ab8c40b7be64e69f30a8892e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4649fd49ab8c40b7be64e69f30a8892e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40177 } } }
I20260809 06:30:35.409806 7005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:35.410390 7006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4649fd49ab8c40b7be64e69f30a8892e. Latest consensus state: current_term: 1 leader_uuid: "4649fd49ab8c40b7be64e69f30a8892e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4649fd49ab8c40b7be64e69f30a8892e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40177 } } }
I20260809 06:30:35.410789 7006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:35.412683 7016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:35.417649 7016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:35.426110 7016 catalog_manager.cc:1347] Generated new cluster ID: 69e893b7d87f4066a1a90de053f10cfe
I20260809 06:30:35.426230 7016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:35.443359 7016 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:35.443974 7016 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:35.460804 7016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4649fd49ab8c40b7be64e69f30a8892e: Generated new TSK 0
I20260809 06:30:35.461892 7016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:35.474071 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 6884
W20260809 06:30:35.476773 6982 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42733
W20260809 06:30:35.477545 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 6884: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:30:35.477721 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 6884
2026-08-09T06:30:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2186 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:35 dist-test-slave-mjjr krb5kdc[7038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:35 dist-test-slave-mjjr krb5kdc[7038](info): set up 2 sockets
Aug 09 06:30:35 dist-test-slave-mjjr krb5kdc[7038](info): commencing operation
krb5kdc: starting...
W20260809 06:30:36.814751 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:37 dist-test-slave-mjjr krb5kdc[7038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:37.142906 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41605
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:37.481971 7056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:37.482249 7056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:37.482450 7056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:37.489835 7056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:37.489938 7056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:37.490052 7056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:37.490149 7056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:37.490214 7056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:37.499912 7056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:37.501148 7056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:37.502481 7056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:37.514659 7063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:37.514577 7064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:37.514703 7070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:37.515683 7056 server_base.cc:1034] running on GCE node
I20260809 06:30:37.516525 7056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:37.517930 7056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:37.519172 7056 hybrid_clock.cc:648] HybridClock initialized: now 1786257037519142 us; error 51 us; skew 500 ppm
I20260809 06:30:37.519542 7056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:37 dist-test-slave-mjjr krb5kdc[7038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257037, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:37.522815 7056 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:37.524470 7056 webserver.cc:460] Webserver started at http://127.0.0.1:46331/ using document root <none> and password file <none>
I20260809 06:30:37.525189 7056 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:37.525347 7056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:37.525664 7056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:37.528842 7056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "dcdef72576974505ada070170f6c9cd9"
format_stamp: "Formatted at 2026-08-09 06:30:37 on dist-test-slave-mjjr"
server_key: "3be639f90bf90112302e6c6fc5da123b"
server_key_iv: "6391449b3e385e06c60ee0e5c9cf8193"
server_key_version: "encryptionkey@0"
I20260809 06:30:37.529702 7056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "dcdef72576974505ada070170f6c9cd9"
format_stamp: "Formatted at 2026-08-09 06:30:37 on dist-test-slave-mjjr"
server_key: "3be639f90bf90112302e6c6fc5da123b"
server_key_iv: "6391449b3e385e06c60ee0e5c9cf8193"
server_key_version: "encryptionkey@0"
I20260809 06:30:37.535054 7056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 06:30:37.538654 7084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:37.540030 7056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:37.540218 7056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "dcdef72576974505ada070170f6c9cd9"
format_stamp: "Formatted at 2026-08-09 06:30:37 on dist-test-slave-mjjr"
server_key: "3be639f90bf90112302e6c6fc5da123b"
server_key_iv: "6391449b3e385e06c60ee0e5c9cf8193"
server_key_version: "encryptionkey@0"
I20260809 06:30:37.540427 7056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:37.570225 7056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:37.574373 7056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:37.574687 7056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:37.593977 7056 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41605
I20260809 06:30:37.594012 7169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41605 every 8 connection(s)
I20260809 06:30:37.595387 7056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:37.599244 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:37.604185 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 7056
I20260809 06:30:37.604406 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:37.604887 8716 external_mini_cluster.cc:1428] Setting key 11cc13d321d32b381a044645eff03811
I20260809 06:30:37.610448 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9: Bootstrap starting.
I20260809 06:30:37.614218 7170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:37.615486 7170 log.cc:824] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:37.619122 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9: No bootstrap required, opened a new log
Aug 09 06:30:37 dist-test-slave-mjjr krb5kdc[7038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 06:30:37.629861 7170 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdef72576974505ada070170f6c9cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41605 } }
I20260809 06:30:37.630355 7170 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:37.630477 7170 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcdef72576974505ada070170f6c9cd9, State: Initialized, Role: FOLLOWER
I20260809 06:30:37.631037 7175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:37.606826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44159 (local address 127.0.0.1:41605)
0809 06:30:37.607789 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:37.607801 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:37.607847 (+ 46us) server_negotiation.cc:407] Connection header received
0809 06:30:37.609565 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:37.609586 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:37.609954 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:37.610259 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:37.612204 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:37.613550 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:37.615511 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:37.616578 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:37.616795 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:37.621663 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:37.621678 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:37.621697 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:37.621743 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:37.625956 (+ 4213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:37.626895 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:37.626904 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:37.626913 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:37.627127 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:37.627600 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:37.627606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:37.627609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:37.628429 (+ 820us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:37.628757 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:37.629185 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 06:30:37.629645 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":436,"threads_started":1}
I20260809 06:30:37.631174 7170 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdef72576974505ada070170f6c9cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41605 } }
I20260809 06:30:37.631494 7170 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:37.631690 7170 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:37.631903 7170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:37.634500 7170 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdef72576974505ada070170f6c9cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41605 } }
I20260809 06:30:37.634977 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:37.635087 7170 leader_election.cc:302] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcdef72576974505ada070170f6c9cd9; no voters:
I20260809 06:30:37.635735 7170 leader_election.cc:288] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:37.635921 7177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:37.636534 7177 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [term 1 LEADER]: Becoming Leader. State: Replica: dcdef72576974505ada070170f6c9cd9, State: Running, Role: LEADER
I20260809 06:30:37.638307 7177 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdef72576974505ada070170f6c9cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41605 } }
I20260809 06:30:37.640321 7177 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcdef72576974505ada070170f6c9cd9. Latest consensus state: current_term: 1 leader_uuid: "dcdef72576974505ada070170f6c9cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdef72576974505ada070170f6c9cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41605 } } }
I20260809 06:30:37.640594 7177 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:37.641168 7179 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcdef72576974505ada070170f6c9cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdef72576974505ada070170f6c9cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41605 } } }
I20260809 06:30:37.641553 7179 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:37.650319 7170 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:37.656433 7185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:37.661896 7185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:37.671140 7175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:37.647929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57137 (local address 127.0.0.1:41605)
0809 06:30:37.648122 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:37.648130 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:37.648887 (+ 757us) server_negotiation.cc:407] Connection header received
0809 06:30:37.649173 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:37.649178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:37.649265 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:37.649340 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:37.651590 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:37.653045 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:37.655450 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:37.656477 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:37.656610 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:37.661031 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:37.661039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:37.661047 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:37.661099 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:37.666076 (+ 4977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:37.667197 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:37.667206 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:37.667213 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:37.667417 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:37.668012 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:37.668020 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:37.668026 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:37.668636 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:37.668943 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:37.670363 (+ 1420us) server_negotiation.cc:299] Negotiation successful
0809 06:30:37.670635 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:30:37.672448 7185 catalog_manager.cc:1347] Generated new cluster ID: dd5ece3e264a4eaeb8fe66234fa28557
I20260809 06:30:37.672585 7185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:37.705282 7185 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:37.706681 7185 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:37.713254 7185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcdef72576974505ada070170f6c9cd9: Generated new TSK 0
I20260809 06:30:37.714275 7185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:37.729009 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7056
I20260809 06:30:37.730118 7162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:37.824795 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 7056
2026-08-09T06:30:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:37 dist-test-slave-mjjr krb5kdc[7220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:37 dist-test-slave-mjjr krb5kdc[7220](info): set up 2 sockets
Aug 09 06:30:37 dist-test-slave-mjjr krb5kdc[7220](info): commencing operation
krb5kdc: starting...
W20260809 06:30:39.161062 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:39 dist-test-slave-mjjr krb5kdc[7220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:39.466723 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:39.817832 7241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:39.818118 7241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:39.818213 7241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:39.825403 7241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:39.825516 7241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:39.825563 7241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:39.825627 7241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:39.825672 7241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:39.835453 7241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:39.836732 7241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:39.838011 7241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:39.850382 7252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:39.850471 7255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:39.850406 7253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:39.851336 7241 server_base.cc:1034] running on GCE node
I20260809 06:30:39.852190 7241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:39.853529 7241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:39.854758 7241 hybrid_clock.cc:648] HybridClock initialized: now 1786257039854698 us; error 30 us; skew 500 ppm
I20260809 06:30:39.855077 7241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:39 dist-test-slave-mjjr krb5kdc[7220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257039, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:39.863024 7241 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:39.864957 7241 webserver.cc:460] Webserver started at http://127.0.0.1:36723/ using document root <none> and password file <none>
I20260809 06:30:39.865933 7241 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:39.866080 7241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:39.866454 7241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:39.870895 7241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "1b3d419799024cf99420aa27e6ca363b"
format_stamp: "Formatted at 2026-08-09 06:30:39 on dist-test-slave-mjjr"
server_key: "161e4de27092b2284352327ad946c182"
server_key_iv: "3b2f2b629537f791638b0ef93a8fc8f3"
server_key_version: "encryptionkey@0"
I20260809 06:30:39.872022 7241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "1b3d419799024cf99420aa27e6ca363b"
format_stamp: "Formatted at 2026-08-09 06:30:39 on dist-test-slave-mjjr"
server_key: "161e4de27092b2284352327ad946c182"
server_key_iv: "3b2f2b629537f791638b0ef93a8fc8f3"
server_key_version: "encryptionkey@0"
I20260809 06:30:39.877202 7241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 06:30:39.880811 7263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:39.882079 7241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 06:30:39.882277 7241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "1b3d419799024cf99420aa27e6ca363b"
format_stamp: "Formatted at 2026-08-09 06:30:39 on dist-test-slave-mjjr"
server_key: "161e4de27092b2284352327ad946c182"
server_key_iv: "3b2f2b629537f791638b0ef93a8fc8f3"
server_key_version: "encryptionkey@0"
I20260809 06:30:39.882483 7241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:39.908228 7241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:39.912395 7241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:39.912705 7241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:39.932256 7241 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34921
I20260809 06:30:39.932303 7345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34921 every 8 connection(s)
I20260809 06:30:39.933611 7241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:39.937420 7346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:39.939832 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 7241
I20260809 06:30:39.940008 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:39.940559 8716 external_mini_cluster.cc:1428] Setting key 3c3467c85ab8980269781850f36ceba8
I20260809 06:30:39.949958 7346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b: Bootstrap starting.
I20260809 06:30:39.953788 7346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:39.954305 7350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:39.942380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57777 (local address 127.0.0.1:34921)
0809 06:30:39.943292 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:39.943303 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:39.943375 (+ 72us) server_negotiation.cc:407] Connection header received
0809 06:30:39.945113 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:39.945135 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:39.945528 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:39.945834 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:39.946483 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:39.947905 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:39.949687 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:39.950439 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:39.950580 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:39.951936 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:39.951944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:39.951958 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:39.952514 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:39.952595 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:39.952620 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:39.952708 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:39.952965 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 06:30:39.953234 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":393,"threads_started":1}
I20260809 06:30:39.954953 7346 log.cc:824] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:39.958155 7346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b: No bootstrap required, opened a new log
I20260809 06:30:39.958304 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:39.968957 7346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3d419799024cf99420aa27e6ca363b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260809 06:30:39.969460 7346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:39.969601 7346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b3d419799024cf99420aa27e6ca363b, State: Initialized, Role: FOLLOWER
I20260809 06:30:39.970291 7346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3d419799024cf99420aa27e6ca363b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260809 06:30:39.970628 7346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:39.970798 7350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:39.963888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38013 (local address 127.0.0.1:34921)
0809 06:30:39.964050 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:39.964054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:39.964390 (+ 336us) server_negotiation.cc:407] Connection header received
0809 06:30:39.964572 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:39.964576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:39.964643 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:39.964727 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:39.965341 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:39.966091 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:39.968004 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:39.968502 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:39.968599 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:39.969874 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:39.969906 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:39.969912 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:39.969936 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:39.969989 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:39.970023 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:39.970118 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:39.970296 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 06:30:39.970364 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 06:30:39.970813 7346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:39.971128 7346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:39.973573 7346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3d419799024cf99420aa27e6ca363b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260809 06:30:39.974079 7346 leader_election.cc:302] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b3d419799024cf99420aa27e6ca363b; no voters:
I20260809 06:30:39.974639 7346 leader_election.cc:288] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:39.974776 7362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:39.975507 7362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [term 1 LEADER]: Becoming Leader. State: Replica: 1b3d419799024cf99420aa27e6ca363b, State: Running, Role: LEADER
I20260809 06:30:39.976032 7362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3d419799024cf99420aa27e6ca363b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260809 06:30:39.976827 7346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:39.978763 7364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b3d419799024cf99420aa27e6ca363b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3d419799024cf99420aa27e6ca363b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } } }
I20260809 06:30:39.979245 7364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:39.979208 7365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b3d419799024cf99420aa27e6ca363b. Latest consensus state: current_term: 1 leader_uuid: "1b3d419799024cf99420aa27e6ca363b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3d419799024cf99420aa27e6ca363b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } } }
I20260809 06:30:39.979713 7365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:39.980916 7368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:39.986452 7368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:39.996317 7368 catalog_manager.cc:1347] Generated new cluster ID: 49e8ff9e6b404b56b38e95be1899c052
I20260809 06:30:39.996445 7368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:40.025017 7368 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:40.026460 7368 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:40.035429 7368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b3d419799024cf99420aa27e6ca363b: Generated new TSK 0
I20260809 06:30:40.036491 7368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:40.043479 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7241
W20260809 06:30:40.044859 7337 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57777
W20260809 06:30:40.045593 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7241: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:30:40.045738 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 7241
2026-08-09T06:30:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2221 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:40 dist-test-slave-mjjr krb5kdc[7389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:40 dist-test-slave-mjjr krb5kdc[7389](info): set up 2 sockets
Aug 09 06:30:40 dist-test-slave-mjjr krb5kdc[7389](info): commencing operation
krb5kdc: starting...
W20260809 06:30:41.379779 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:41 dist-test-slave-mjjr krb5kdc[7389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:41.675387 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:42.017869 7408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:42.018222 7408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:42.018419 7408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:42.025493 7408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:42.025665 7408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:42.025799 7408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:42.025957 7408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:42.026046 7408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:42.035684 7408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:42.036994 7408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:42.038300 7408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:42.050185 7416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:42.050158 7419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:42.050635 7417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:42.050978 7408 server_base.cc:1034] running on GCE node
I20260809 06:30:42.051820 7408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:42.053115 7408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:42.054322 7408 hybrid_clock.cc:648] HybridClock initialized: now 1786257042054265 us; error 48 us; skew 500 ppm
I20260809 06:30:42.054674 7408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:42 dist-test-slave-mjjr krb5kdc[7389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:42.057828 7408 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:42.059760 7408 webserver.cc:460] Webserver started at http://127.0.0.1:35929/ using document root <none> and password file <none>
I20260809 06:30:42.060453 7408 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:42.060606 7408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:42.060940 7408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:42.063994 7408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "ab1028d78d364981a47906ffa7971673"
format_stamp: "Formatted at 2026-08-09 06:30:42 on dist-test-slave-mjjr"
server_key: "bd9a828114c110437460bd0a13057cab"
server_key_iv: "0d40d4325e1262c79460f56039eb95d9"
server_key_version: "encryptionkey@0"
I20260809 06:30:42.064775 7408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "ab1028d78d364981a47906ffa7971673"
format_stamp: "Formatted at 2026-08-09 06:30:42 on dist-test-slave-mjjr"
server_key: "bd9a828114c110437460bd0a13057cab"
server_key_iv: "0d40d4325e1262c79460f56039eb95d9"
server_key_version: "encryptionkey@0"
I20260809 06:30:42.069869 7408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:30:42.073269 7429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:42.074509 7408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 06:30:42.074700 7408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "ab1028d78d364981a47906ffa7971673"
format_stamp: "Formatted at 2026-08-09 06:30:42 on dist-test-slave-mjjr"
server_key: "bd9a828114c110437460bd0a13057cab"
server_key_iv: "0d40d4325e1262c79460f56039eb95d9"
server_key_version: "encryptionkey@0"
I20260809 06:30:42.074911 7408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:42.080590 7408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:42.084514 7408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:42.084816 7408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:42.103789 7408 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40201
I20260809 06:30:42.103854 7517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40201 every 8 connection(s)
I20260809 06:30:42.105145 7408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:42.108860 7518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:42.114723 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 7408
I20260809 06:30:42.114923 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:42.115490 8716 external_mini_cluster.cc:1428] Setting key 97b0a8ab3eeb3a695e4a9720392f5681
I20260809 06:30:42.119894 7518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673: Bootstrap starting.
I20260809 06:30:42.123425 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:42.117423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41673 (local address 127.0.0.1:40201)
0809 06:30:42.118282 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:42.118293 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:42.118333 (+ 40us) server_negotiation.cc:407] Connection header received
0809 06:30:42.120175 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:42.120197 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:42.120517 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:42.120821 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:42.121928 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:42.121934 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:42.121945 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:42.121959 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:42.122027 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:42.122048 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:42.122211 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:42.122313 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 06:30:42.122551 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":400,"threads_started":1}
I20260809 06:30:42.123522 7518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:42.124666 7518 log.cc:824] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:42.127381 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:42.127872 7518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673: No bootstrap required, opened a new log
I20260809 06:30:42.134347 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:42.131940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48521 (local address 127.0.0.1:40201)
0809 06:30:42.132112 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:42.132118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:42.132506 (+ 388us) server_negotiation.cc:407] Connection header received
0809 06:30:42.132635 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:42.132640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:42.132702 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:42.132813 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:42.133713 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:42.133724 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:42.133729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:42.133746 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:42.133796 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:42.133817 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:42.133959 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:42.134005 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 06:30:42.134070 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:30:42.136138 7518 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1028d78d364981a47906ffa7971673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260809 06:30:42.136510 7518 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:42.136608 7518 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab1028d78d364981a47906ffa7971673, State: Initialized, Role: FOLLOWER
I20260809 06:30:42.137250 7518 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1028d78d364981a47906ffa7971673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260809 06:30:42.137506 7518 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:42.137666 7518 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:42.137847 7518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:42.140301 7518 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1028d78d364981a47906ffa7971673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260809 06:30:42.140822 7518 leader_election.cc:302] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab1028d78d364981a47906ffa7971673; no voters:
I20260809 06:30:42.141448 7518 leader_election.cc:288] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:42.141654 7536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:42.143527 7518 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:42.148556 7536 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [term 1 LEADER]: Becoming Leader. State: Replica: ab1028d78d364981a47906ffa7971673, State: Running, Role: LEADER
I20260809 06:30:42.149443 7536 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1028d78d364981a47906ffa7971673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260809 06:30:42.152542 7537 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab1028d78d364981a47906ffa7971673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1028d78d364981a47906ffa7971673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } } }
I20260809 06:30:42.152828 7538 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab1028d78d364981a47906ffa7971673. Latest consensus state: current_term: 1 leader_uuid: "ab1028d78d364981a47906ffa7971673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1028d78d364981a47906ffa7971673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } } }
I20260809 06:30:42.153021 7537 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:42.153128 7538 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:42.154487 7558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:42.159727 7558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:42.168452 7558 catalog_manager.cc:1347] Generated new cluster ID: 3135471b0131417ab8cf2e72c888dd17
I20260809 06:30:42.168560 7558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:42.185319 7558 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:42.185806 7558 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:42.192373 7558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab1028d78d364981a47906ffa7971673: Generated new TSK 0
I20260809 06:30:42.193293 7558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:42.198663 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7408
W20260809 06:30:42.199731 7502 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41673
W20260809 06:30:42.200654 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7408: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:30:42.200853 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 7408
2026-08-09T06:30:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2153 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:42 dist-test-slave-mjjr krb5kdc[7571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:42 dist-test-slave-mjjr krb5kdc[7571](info): set up 2 sockets
Aug 09 06:30:42 dist-test-slave-mjjr krb5kdc[7571](info): commencing operation
krb5kdc: starting...
W20260809 06:30:43.522337 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:43 dist-test-slave-mjjr krb5kdc[7571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:43.835551 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:44.186813 7592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:44.187155 7592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:44.187323 7592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:44.194571 7592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:44.194726 7592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:44.194839 7592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:44.194959 7592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:44.195051 7592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:44.205416 7592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:44.206791 7592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:44.208160 7592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:44.220026 7607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:44.220058 7600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:44.220482 7603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:44.220849 7592 server_base.cc:1034] running on GCE node
I20260809 06:30:44.221601 7592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:44.222880 7592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:44.224125 7592 hybrid_clock.cc:648] HybridClock initialized: now 1786257044224063 us; error 49 us; skew 500 ppm
I20260809 06:30:44.224478 7592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:44 dist-test-slave-mjjr krb5kdc[7571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:44.227741 7592 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:44.229147 7592 webserver.cc:460] Webserver started at http://127.0.0.1:43065/ using document root <none> and password file <none>
I20260809 06:30:44.229820 7592 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:44.229988 7592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:44.230295 7592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:44.233256 7592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "e68c583487464a99936646e6e3f31f19"
format_stamp: "Formatted at 2026-08-09 06:30:44 on dist-test-slave-mjjr"
server_key: "437885c35046a27158ca258e11db91e7"
server_key_iv: "990e91f7244ea3718020caf9bd500028"
server_key_version: "encryptionkey@0"
I20260809 06:30:44.234059 7592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "e68c583487464a99936646e6e3f31f19"
format_stamp: "Formatted at 2026-08-09 06:30:44 on dist-test-slave-mjjr"
server_key: "437885c35046a27158ca258e11db91e7"
server_key_iv: "990e91f7244ea3718020caf9bd500028"
server_key_version: "encryptionkey@0"
I20260809 06:30:44.239192 7592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 06:30:44.242568 7615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:44.243916 7592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 06:30:44.244118 7592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "e68c583487464a99936646e6e3f31f19"
format_stamp: "Formatted at 2026-08-09 06:30:44 on dist-test-slave-mjjr"
server_key: "437885c35046a27158ca258e11db91e7"
server_key_iv: "990e91f7244ea3718020caf9bd500028"
server_key_version: "encryptionkey@0"
I20260809 06:30:44.244310 7592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:44.274041 7592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:44.278244 7592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:44.278571 7592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:44.298316 7592 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37311
I20260809 06:30:44.298369 7695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37311 every 8 connection(s)
I20260809 06:30:44.299718 7592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:44.303416 7696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:44.307269 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 7592
I20260809 06:30:44.307462 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:44.308077 8716 external_mini_cluster.cc:1428] Setting key 6952afe97a6c885b72e00fa43bf1bbcd
I20260809 06:30:44.316769 7696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19: Bootstrap starting.
I20260809 06:30:44.320688 7696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:44.322113 7696 log.cc:824] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19: Log is configured to *not* fsync() on all Append() calls
Aug 09 06:30:44 dist-test-slave-mjjr krb5kdc[7571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 06:30:44.325222 7696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19: No bootstrap required, opened a new log
I20260809 06:30:44.332042 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:44.309985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:43209 (local address 127.0.0.1:37311)
0809 06:30:44.310765 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:44.310776 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:44.310816 (+ 40us) server_negotiation.cc:407] Connection header received
0809 06:30:44.312527 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:44.312549 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:44.312943 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:44.313259 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:44.315150 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:44.316553 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:44.318339 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:44.319004 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:44.319132 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:44.324227 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:44.324235 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:44.324249 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:44.324285 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:44.327556 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:44.328478 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:44.328485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:44.328491 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:44.328668 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:44.329207 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:44.329211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:44.329213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:44.329759 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:44.330105 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:44.330646 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 06:30:44.330961 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":371,"threads_started":1}
I20260809 06:30:44.333225 7696 raft_consensus.cc:348] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68c583487464a99936646e6e3f31f19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37311 } }
I20260809 06:30:44.333586 7696 raft_consensus.cc:374] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:44.333694 7696 raft_consensus.cc:729] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68c583487464a99936646e6e3f31f19, State: Initialized, Role: FOLLOWER
I20260809 06:30:44.334285 7696 consensus_queue.cc:260] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68c583487464a99936646e6e3f31f19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37311 } }
I20260809 06:30:44.334560 7696 raft_consensus.cc:388] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:44.334733 7696 raft_consensus.cc:482] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:44.334918 7696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:44.336213 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:44.337970 7696 raft_consensus.cc:504] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68c583487464a99936646e6e3f31f19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37311 } }
I20260809 06:30:44.338515 7696 leader_election.cc:302] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e68c583487464a99936646e6e3f31f19; no voters:
I20260809 06:30:44.339179 7696 leader_election.cc:288] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:44.339298 7703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:44.341758 7696 sys_catalog.cc:564] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:44.344017 7703 raft_consensus.cc:686] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [term 1 LEADER]: Becoming Leader. State: Replica: e68c583487464a99936646e6e3f31f19, State: Running, Role: LEADER
I20260809 06:30:44.344743 7703 consensus_queue.cc:237] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68c583487464a99936646e6e3f31f19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37311 } }
I20260809 06:30:44.348340 7705 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e68c583487464a99936646e6e3f31f19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68c583487464a99936646e6e3f31f19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37311 } } }
I20260809 06:30:44.348724 7705 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:44.349398 7707 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e68c583487464a99936646e6e3f31f19. Latest consensus state: current_term: 1 leader_uuid: "e68c583487464a99936646e6e3f31f19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68c583487464a99936646e6e3f31f19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37311 } } }
I20260809 06:30:44.349748 7707 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:44.349763 7728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:44.361368 7728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:44.374357 7728 catalog_manager.cc:1347] Generated new cluster ID: a16fb47c3f624da18517eea25365761e
I20260809 06:30:44.374485 7728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:44.377926 7700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:44.354088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:59689 (local address 127.0.0.1:37311)
0809 06:30:44.354306 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:44.354315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:44.356512 (+ 2197us) server_negotiation.cc:407] Connection header received
0809 06:30:44.360298 (+ 3786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:44.360304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:44.360389 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:44.360465 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:44.362658 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:44.363950 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:44.365961 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:44.366896 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:44.367076 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:44.370836 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:44.370842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:44.370845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:44.370879 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:44.374416 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:44.375449 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:44.375457 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:44.375462 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:44.375626 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:44.376255 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:44.376260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:44.376264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:44.376673 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:44.376967 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:44.377341 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 06:30:44.377528 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":65}
I20260809 06:30:44.399873 7728 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:44.401942 7728 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:44.411584 7728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e68c583487464a99936646e6e3f31f19: Generated new TSK 0
I20260809 06:30:44.412714 7728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:44.422191 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7592
I20260809 06:30:44.423767 7685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:44.516811 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 7592
W20260809 06:30:44.541960 7568 connection.cc:535] client connection to 127.0.0.1:37311 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37311): Connection reset by peer (error 104)
2026-08-09T06:30:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:44 dist-test-slave-mjjr krb5kdc[7751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:44 dist-test-slave-mjjr krb5kdc[7751](info): set up 2 sockets
Aug 09 06:30:44 dist-test-slave-mjjr krb5kdc[7751](info): commencing operation
krb5kdc: starting...
W20260809 06:30:45.859992 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:46 dist-test-slave-mjjr krb5kdc[7751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:46.179586 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34655
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:46.517632 7774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:46.517936 7774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:46.518074 7774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:46.525144 7774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:46.525251 7774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:46.525298 7774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:46.525410 7774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:46.525502 7774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:46.535085 7774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:46.536352 7774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:46.537652 7774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:46.550529 7784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:46.550575 7787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:46.550630 7785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:46.553069 7774 server_base.cc:1034] running on GCE node
I20260809 06:30:46.553859 7774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:46.555320 7774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:46.556623 7774 hybrid_clock.cc:648] HybridClock initialized: now 1786257046556578 us; error 68 us; skew 500 ppm
I20260809 06:30:46.557052 7774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:46 dist-test-slave-mjjr krb5kdc[7751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257046, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:46.560287 7774 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:46.561935 7774 webserver.cc:460] Webserver started at http://127.0.0.1:37541/ using document root <none> and password file <none>
I20260809 06:30:46.562654 7774 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:46.562824 7774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:46.563149 7774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:46.566182 7774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "b7fe6a720d5746f3843c1d55c224b86c"
format_stamp: "Formatted at 2026-08-09 06:30:46 on dist-test-slave-mjjr"
server_key: "9409d86603f4c54e64c626119ceb368c"
server_key_iv: "80cd0df34d4bdb1401fe5fb2213e461c"
server_key_version: "encryptionkey@0"
I20260809 06:30:46.567014 7774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "b7fe6a720d5746f3843c1d55c224b86c"
format_stamp: "Formatted at 2026-08-09 06:30:46 on dist-test-slave-mjjr"
server_key: "9409d86603f4c54e64c626119ceb368c"
server_key_iv: "80cd0df34d4bdb1401fe5fb2213e461c"
server_key_version: "encryptionkey@0"
I20260809 06:30:46.572132 7774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 06:30:46.575496 7801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:46.576954 7774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:46.577169 7774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "b7fe6a720d5746f3843c1d55c224b86c"
format_stamp: "Formatted at 2026-08-09 06:30:46 on dist-test-slave-mjjr"
server_key: "9409d86603f4c54e64c626119ceb368c"
server_key_iv: "80cd0df34d4bdb1401fe5fb2213e461c"
server_key_version: "encryptionkey@0"
I20260809 06:30:46.577387 7774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:46.614602 7774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:46.618667 7774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:46.618968 7774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:46.638147 7774 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34655
I20260809 06:30:46.638191 7883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34655 every 8 connection(s)
I20260809 06:30:46.639561 7774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:46.640115 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 7774
I20260809 06:30:46.640316 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:46.640944 8716 external_mini_cluster.cc:1428] Setting key be23f24c29deef644eec0c3bb6c11ca6
I20260809 06:30:46.644047 7885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:46.654007 7888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:46.642760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:52479 (local address 127.0.0.1:34655)
0809 06:30:46.643538 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:46.643550 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:46.643589 (+ 39us) server_negotiation.cc:407] Connection header received
0809 06:30:46.645409 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:46.645432 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:46.645810 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:46.646222 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:46.646821 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:46.648210 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:46.649541 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:46.650196 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:46.650409 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:46.651783 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:46.651791 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:46.651804 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:46.652357 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:46.652433 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:46.652461 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:46.652645 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:46.652944 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 06:30:46.653115 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":336,"threads_started":1}
I20260809 06:30:46.654855 7885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c: Bootstrap starting.
I20260809 06:30:46.658043 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:46.658609 7885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:46.659581 7885 log.cc:824] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:46.662635 7885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c: No bootstrap required, opened a new log
I20260809 06:30:46.669646 7888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:46.662642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:50231 (local address 127.0.0.1:34655)
0809 06:30:46.662858 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:46.662866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:46.663328 (+ 462us) server_negotiation.cc:407] Connection header received
0809 06:30:46.663521 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:46.663525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:46.663586 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:46.663727 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:46.664309 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:46.665090 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:46.667100 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:46.667523 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:46.667615 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:46.668833 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:46.668837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:46.668841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:46.668855 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:46.668905 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:46.668923 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:46.669015 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:46.669231 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 06:30:46.669299 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 06:30:46.672561 7885 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fe6a720d5746f3843c1d55c224b86c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } }
I20260809 06:30:46.672936 7885 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:46.673053 7885 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fe6a720d5746f3843c1d55c224b86c, State: Initialized, Role: FOLLOWER
I20260809 06:30:46.673602 7885 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fe6a720d5746f3843c1d55c224b86c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } }
I20260809 06:30:46.673851 7885 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:46.673995 7885 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:46.674187 7885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:46.676821 7885 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fe6a720d5746f3843c1d55c224b86c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } }
I20260809 06:30:46.677276 7885 leader_election.cc:302] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7fe6a720d5746f3843c1d55c224b86c; no voters:
I20260809 06:30:46.677774 7885 leader_election.cc:288] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:46.677956 7899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:46.678639 7899 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [term 1 LEADER]: Becoming Leader. State: Replica: b7fe6a720d5746f3843c1d55c224b86c, State: Running, Role: LEADER
I20260809 06:30:46.679124 7899 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fe6a720d5746f3843c1d55c224b86c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } }
I20260809 06:30:46.679965 7885 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:46.681996 7901 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7fe6a720d5746f3843c1d55c224b86c. Latest consensus state: current_term: 1 leader_uuid: "b7fe6a720d5746f3843c1d55c224b86c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fe6a720d5746f3843c1d55c224b86c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } } }
I20260809 06:30:46.681874 7900 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7fe6a720d5746f3843c1d55c224b86c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fe6a720d5746f3843c1d55c224b86c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } } }
I20260809 06:30:46.682515 7901 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:46.682525 7900 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:46.683518 7905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:46.690625 7905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:46.699689 7905 catalog_manager.cc:1347] Generated new cluster ID: a640b9512a0f47d3894f3e80c64c0034
I20260809 06:30:46.699846 7905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:46.715147 7905 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:46.716442 7905 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:46.725121 7905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7fe6a720d5746f3843c1d55c224b86c: Generated new TSK 0
I20260809 06:30:46.725862 7905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:46.739015 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7774
W20260809 06:30:46.740130 7875 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.116:52479
W20260809 06:30:46.740933 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7774: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:30:46.741101 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 7774
W20260809 06:30:46.768682 7749 connection.cc:535] client connection to 127.0.0.1:34655 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34655): Connection reset by peer (error 104)
2026-08-09T06:30:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2226 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:46 dist-test-slave-mjjr krb5kdc[7929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:46 dist-test-slave-mjjr krb5kdc[7929](info): set up 2 sockets
Aug 09 06:30:46 dist-test-slave-mjjr krb5kdc[7929](info): commencing operation
krb5kdc: starting...
W20260809 06:30:48.067068 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:48 dist-test-slave-mjjr krb5kdc[7929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:48.378396 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46877
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:48.715160 7955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:48.715507 7955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:48.715704 7955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:48.722797 7955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:48.722980 7955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:48.723085 7955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:48.723197 7955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:48.723289 7955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:48.733174 7955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:48.734483 7955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:48.735795 7955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:48.747956 7965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:48.748008 7963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:48.747903 7970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:48.749078 7955 server_base.cc:1034] running on GCE node
I20260809 06:30:48.749859 7955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:48.751263 7955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:48.752498 7955 hybrid_clock.cc:648] HybridClock initialized: now 1786257048752463 us; error 50 us; skew 500 ppm
I20260809 06:30:48.752857 7955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:48 dist-test-slave-mjjr krb5kdc[7929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257048, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:48.756105 7955 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:48.757746 7955 webserver.cc:460] Webserver started at http://127.0.0.1:45167/ using document root <none> and password file <none>
I20260809 06:30:48.758450 7955 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:48.758603 7955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:48.758930 7955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:48.762717 7955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "4a4dd9f3e8914f669ee97de38239e9e1"
format_stamp: "Formatted at 2026-08-09 06:30:48 on dist-test-slave-mjjr"
server_key: "4cb913777d9f5d2963bf0dc4dfb48a25"
server_key_iv: "08011578948f4ba86e8edc13aa57743a"
server_key_version: "encryptionkey@0"
I20260809 06:30:48.763729 7955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "4a4dd9f3e8914f669ee97de38239e9e1"
format_stamp: "Formatted at 2026-08-09 06:30:48 on dist-test-slave-mjjr"
server_key: "4cb913777d9f5d2963bf0dc4dfb48a25"
server_key_iv: "08011578948f4ba86e8edc13aa57743a"
server_key_version: "encryptionkey@0"
I20260809 06:30:48.768945 7955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 06:30:48.772385 7983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:48.773692 7955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:30:48.773838 7955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "4a4dd9f3e8914f669ee97de38239e9e1"
format_stamp: "Formatted at 2026-08-09 06:30:48 on dist-test-slave-mjjr"
server_key: "4cb913777d9f5d2963bf0dc4dfb48a25"
server_key_iv: "08011578948f4ba86e8edc13aa57743a"
server_key_version: "encryptionkey@0"
I20260809 06:30:48.774073 7955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:48.780062 7955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:48.784425 7955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:48.784736 7955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:48.803719 7955 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46877
I20260809 06:30:48.803845 8055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46877 every 8 connection(s)
I20260809 06:30:48.805123 7955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:48.807689 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 7955
I20260809 06:30:48.807914 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:48.808499 8716 external_mini_cluster.cc:1428] Setting key 6693395d57b57703499527eef59ea00f
I20260809 06:30:48.809763 8056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:48.817044 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:48.810282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:41939 (local address 127.0.0.1:46877)
0809 06:30:48.811080 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:48.811096 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:48.811147 (+ 51us) server_negotiation.cc:407] Connection header received
0809 06:30:48.813166 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:48.813194 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:48.813667 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:48.814090 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:48.815063 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:48.815073 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:48.815093 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:48.815110 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:48.815234 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:48.815269 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:48.815477 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:48.815609 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 06:30:48.815821 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":329,"threads_started":1}
I20260809 06:30:48.820946 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:48.823784 8056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1: Bootstrap starting.
I20260809 06:30:48.827665 8056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:48.828203 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:48.825464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:56993 (local address 127.0.0.1:46877)
0809 06:30:48.825653 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:48.825659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:48.826203 (+ 544us) server_negotiation.cc:407] Connection header received
0809 06:30:48.826370 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:48.826374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:48.826423 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:48.826515 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:48.827472 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:48.827476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:48.827479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:48.827493 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:48.827532 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:48.827545 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:48.827631 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:48.827847 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 06:30:48.827901 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 06:30:48.829034 8056 log.cc:824] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:48.831763 8056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1: No bootstrap required, opened a new log
I20260809 06:30:48.838986 8056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4dd9f3e8914f669ee97de38239e9e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46877 } }
I20260809 06:30:48.839326 8056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:48.839397 8056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a4dd9f3e8914f669ee97de38239e9e1, State: Initialized, Role: FOLLOWER
I20260809 06:30:48.839941 8056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4dd9f3e8914f669ee97de38239e9e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46877 } }
I20260809 06:30:48.840150 8056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:48.840252 8056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:48.840376 8056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:48.842585 8056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4dd9f3e8914f669ee97de38239e9e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46877 } }
I20260809 06:30:48.843027 8056 leader_election.cc:302] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a4dd9f3e8914f669ee97de38239e9e1; no voters:
I20260809 06:30:48.843534 8056 leader_election.cc:288] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:48.843708 8074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:48.844587 8074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [term 1 LEADER]: Becoming Leader. State: Replica: 4a4dd9f3e8914f669ee97de38239e9e1, State: Running, Role: LEADER
I20260809 06:30:48.845115 8074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4dd9f3e8914f669ee97de38239e9e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46877 } }
I20260809 06:30:48.845858 8056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:48.847987 8075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a4dd9f3e8914f669ee97de38239e9e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4dd9f3e8914f669ee97de38239e9e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46877 } } }
I20260809 06:30:48.848121 8077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a4dd9f3e8914f669ee97de38239e9e1. Latest consensus state: current_term: 1 leader_uuid: "4a4dd9f3e8914f669ee97de38239e9e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4dd9f3e8914f669ee97de38239e9e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46877 } } }
I20260809 06:30:48.848429 8075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:48.848526 8077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:48.852795 8081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:48.858381 8081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:48.866531 8081 catalog_manager.cc:1347] Generated new cluster ID: 03ca5261a35148a68a0d861ce83d6e0d
I20260809 06:30:48.866685 8081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:48.918645 8081 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:48.919131 8081 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:48.926448 8081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a4dd9f3e8914f669ee97de38239e9e1: Generated new TSK 0
I20260809 06:30:48.927255 8081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:48.942199 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7955
W20260809 06:30:48.943320 8045 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.116:41939
W20260809 06:30:48.944113 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 7955: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:30:48.944267 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 7955
2026-08-09T06:30:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2200 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:49 dist-test-slave-mjjr krb5kdc[8100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:49 dist-test-slave-mjjr krb5kdc[8100](info): set up 2 sockets
Aug 09 06:30:49 dist-test-slave-mjjr krb5kdc[8100](info): commencing operation
krb5kdc: starting...
W20260809 06:30:50.268718 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:50 dist-test-slave-mjjr krb5kdc[8100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:50.562413 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:50.916567 8129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:50.916935 8129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:50.917098 8129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:50.925166 8129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:50.925362 8129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:50.925486 8129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:50.925611 8129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:50.925710 8129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:50.936210 8129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:50.937696 8129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:50.939159 8129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:50.952064 8141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:50.952155 8134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:50.952155 8136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:50.953172 8129 server_base.cc:1034] running on GCE node
I20260809 06:30:50.954023 8129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:50.955559 8129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:50.956844 8129 hybrid_clock.cc:648] HybridClock initialized: now 1786257050956789 us; error 42 us; skew 500 ppm
I20260809 06:30:50.957237 8129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:50 dist-test-slave-mjjr krb5kdc[8100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257050, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:50.963877 8129 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:50.965502 8129 webserver.cc:460] Webserver started at http://127.0.0.1:36033/ using document root <none> and password file <none>
I20260809 06:30:50.966264 8129 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:50.966419 8129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:50.966768 8129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:50.969827 8129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "5f2fbf70f47041f1a15f8c29d03f7038"
format_stamp: "Formatted at 2026-08-09 06:30:50 on dist-test-slave-mjjr"
server_key: "5e5cee383b3368c9dfb1d9ee7f275865"
server_key_iv: "05423aab91deabb091207265424deb6a"
server_key_version: "encryptionkey@0"
I20260809 06:30:50.970615 8129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "5f2fbf70f47041f1a15f8c29d03f7038"
format_stamp: "Formatted at 2026-08-09 06:30:50 on dist-test-slave-mjjr"
server_key: "5e5cee383b3368c9dfb1d9ee7f275865"
server_key_iv: "05423aab91deabb091207265424deb6a"
server_key_version: "encryptionkey@0"
I20260809 06:30:50.978287 8129 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260809 06:30:50.982610 8153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:50.983989 8129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:50.984189 8129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "5f2fbf70f47041f1a15f8c29d03f7038"
format_stamp: "Formatted at 2026-08-09 06:30:50 on dist-test-slave-mjjr"
server_key: "5e5cee383b3368c9dfb1d9ee7f275865"
server_key_iv: "05423aab91deabb091207265424deb6a"
server_key_version: "encryptionkey@0"
I20260809 06:30:50.984375 8129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:51.008210 8129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:51.012151 8129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:51.012444 8129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:51.031983 8129 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32977
I20260809 06:30:51.032018 8223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32977 every 8 connection(s)
I20260809 06:30:51.033375 8129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:51.037012 8225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:51.041899 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 8129
I20260809 06:30:51.042086 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:51.042662 8716 external_mini_cluster.cc:1428] Setting key 7476c412111942e3f59bf3c4550d724f
I20260809 06:30:51.048419 8225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038: Bootstrap starting.
I20260809 06:30:51.052027 8225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:51.053233 8225 log.cc:824] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:51.056768 8225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038: No bootstrap required, opened a new log
Aug 09 06:30:51 dist-test-slave-mjjr krb5kdc[8100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 06:30:51.064585 8225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2fbf70f47041f1a15f8c29d03f7038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32977 } }
I20260809 06:30:51.064965 8225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:51.065099 8225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2fbf70f47041f1a15f8c29d03f7038, State: Initialized, Role: FOLLOWER
I20260809 06:30:51.065753 8225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2fbf70f47041f1a15f8c29d03f7038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32977 } }
I20260809 06:30:51.066054 8225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:51.066216 8225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:51.066408 8225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:51.066679 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:51.044503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:53955 (local address 127.0.0.1:32977)
0809 06:30:51.045308 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:51.045319 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:51.045363 (+ 44us) server_negotiation.cc:407] Connection header received
0809 06:30:51.047122 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:51.047146 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:51.047509 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:51.047892 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:51.049720 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:51.051056 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:51.052870 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:51.053581 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:51.053769 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:51.058681 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:51.058688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:51.058702 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:51.058745 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:51.062039 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:51.062922 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:51.062928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:51.062934 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:51.063106 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:51.063621 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:51.063624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:51.063626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:51.064204 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:51.064537 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:51.065070 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 06:30:51.065385 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":375,"threads_started":1}
I20260809 06:30:51.068964 8225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2fbf70f47041f1a15f8c29d03f7038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32977 } }
I20260809 06:30:51.069491 8225 leader_election.cc:302] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f2fbf70f47041f1a15f8c29d03f7038; no voters:
I20260809 06:30:51.070186 8225 leader_election.cc:288] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:51.070323 8233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:51.070653 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:51.071983 8233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [term 1 LEADER]: Becoming Leader. State: Replica: 5f2fbf70f47041f1a15f8c29d03f7038, State: Running, Role: LEADER
I20260809 06:30:51.072588 8225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:51.072624 8233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2fbf70f47041f1a15f8c29d03f7038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32977 } }
I20260809 06:30:51.075769 8236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f2fbf70f47041f1a15f8c29d03f7038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2fbf70f47041f1a15f8c29d03f7038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32977 } } }
I20260809 06:30:51.076186 8236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:51.076776 8237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f2fbf70f47041f1a15f8c29d03f7038. Latest consensus state: current_term: 1 leader_uuid: "5f2fbf70f47041f1a15f8c29d03f7038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f2fbf70f47041f1a15f8c29d03f7038" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32977 } } }
I20260809 06:30:51.077988 8237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:51.087381 8248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:51.092689 8248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:51.097524 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:51.076407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:51379 (local address 127.0.0.1:32977)
0809 06:30:51.076628 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:51.076636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:51.077340 (+ 704us) server_negotiation.cc:407] Connection header received
0809 06:30:51.077641 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:51.077649 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:51.077753 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:51.077888 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:51.079746 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:51.081463 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:51.085555 (+ 4092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:51.085973 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:51.086076 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:51.089852 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:51.089859 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:51.089864 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:51.089908 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:51.093986 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:51.094999 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:51.095004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:51.095008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:51.095152 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:51.095748 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:51.095754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:51.095757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:51.096191 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:51.096473 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:51.096980 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 06:30:51.097153 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 06:30:51.102119 8248 catalog_manager.cc:1347] Generated new cluster ID: 414e43399be34213aeccc1825cd9c524
I20260809 06:30:51.102277 8248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:51.115052 8248 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:51.116339 8248 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:51.125208 8248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f2fbf70f47041f1a15f8c29d03f7038: Generated new TSK 0
I20260809 06:30:51.126188 8248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:51.137205 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8129
I20260809 06:30:51.138365 8218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:51.228332 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 8129
W20260809 06:30:51.251011 8098 connection.cc:535] client connection to 127.0.0.1:32977 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32977): Connection reset by peer (error 104)
2026-08-09T06:30:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2281 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:51 dist-test-slave-mjjr krb5kdc[8282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:51 dist-test-slave-mjjr krb5kdc[8282](info): set up 2 sockets
Aug 09 06:30:51 dist-test-slave-mjjr krb5kdc[8282](info): commencing operation
krb5kdc: starting...
W20260809 06:30:52.558457 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:52 dist-test-slave-mjjr krb5kdc[8282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:52.869134 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:53.209558 8310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:53.209913 8310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:53.210081 8310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:53.217339 8310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:53.217492 8310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:53.217602 8310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:53.217710 8310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:53.217794 8310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:53.227826 8310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:53.229126 8310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:53.230410 8310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:53.242216 8323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:53.242188 8316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:53.242313 8318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:53.243139 8310 server_base.cc:1034] running on GCE node
I20260809 06:30:53.243952 8310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:53.245359 8310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:53.246592 8310 hybrid_clock.cc:648] HybridClock initialized: now 1786257053246562 us; error 51 us; skew 500 ppm
I20260809 06:30:53.246948 8310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:53 dist-test-slave-mjjr krb5kdc[8282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257053, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:53.250126 8310 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:53.251564 8310 webserver.cc:460] Webserver started at http://127.0.0.1:38195/ using document root <none> and password file <none>
I20260809 06:30:53.252274 8310 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:53.252424 8310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:53.252760 8310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:53.255774 8310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "11a79221473748f1ac794d89a6ec7379"
format_stamp: "Formatted at 2026-08-09 06:30:53 on dist-test-slave-mjjr"
server_key: "f3ced680b9dc273403100bc067b20c20"
server_key_iv: "2cd7eb7a6d72e5a4e8ca045fb75e6887"
server_key_version: "encryptionkey@0"
I20260809 06:30:53.256501 8310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "11a79221473748f1ac794d89a6ec7379"
format_stamp: "Formatted at 2026-08-09 06:30:53 on dist-test-slave-mjjr"
server_key: "f3ced680b9dc273403100bc067b20c20"
server_key_iv: "2cd7eb7a6d72e5a4e8ca045fb75e6887"
server_key_version: "encryptionkey@0"
I20260809 06:30:53.261500 8310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 06:30:53.265022 8336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:53.266292 8310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:30:53.266433 8310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "11a79221473748f1ac794d89a6ec7379"
format_stamp: "Formatted at 2026-08-09 06:30:53 on dist-test-slave-mjjr"
server_key: "f3ced680b9dc273403100bc067b20c20"
server_key_iv: "2cd7eb7a6d72e5a4e8ca045fb75e6887"
server_key_version: "encryptionkey@0"
I20260809 06:30:53.266608 8310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:53.282697 8310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:53.286746 8310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:53.286980 8310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:53.305881 8310 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39719
I20260809 06:30:53.305928 8426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39719 every 8 connection(s)
I20260809 06:30:53.307224 8310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:53.310884 8429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:53.313791 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 8310
I20260809 06:30:53.314005 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:53.314555 8716 external_mini_cluster.cc:1428] Setting key d9e4fcaa93f60d1e293a21ea4d98260a
I20260809 06:30:53.322795 8429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379: Bootstrap starting.
I20260809 06:30:53.326563 8429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:53.327802 8429 log.cc:824] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:53.327919 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:53.316380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:36329 (local address 127.0.0.1:39719)
0809 06:30:53.317159 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:53.317170 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:53.317208 (+ 38us) server_negotiation.cc:407] Connection header received
0809 06:30:53.318865 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:53.318885 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:53.319253 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:53.319557 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:53.320171 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:53.321477 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:53.323403 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:53.324116 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:53.324250 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:53.325584 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:53.325594 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:53.325612 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:53.326151 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:53.326225 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:53.326249 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:53.326367 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:53.326734 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 06:30:53.326893 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":360,"threads_started":1}
I20260809 06:30:53.330763 8429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379: No bootstrap required, opened a new log
I20260809 06:30:53.331564 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:53.338325 8429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a79221473748f1ac794d89a6ec7379" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39719 } }
I20260809 06:30:53.338719 8429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:53.338855 8429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11a79221473748f1ac794d89a6ec7379, State: Initialized, Role: FOLLOWER
I20260809 06:30:53.339493 8429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a79221473748f1ac794d89a6ec7379" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39719 } }
I20260809 06:30:53.339826 8429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:53.339993 8429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:53.340183 8429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:53.343218 8429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a79221473748f1ac794d89a6ec7379" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39719 } }
I20260809 06:30:53.343899 8429 leader_election.cc:302] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11a79221473748f1ac794d89a6ec7379; no voters:
I20260809 06:30:53.344606 8429 leader_election.cc:288] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:53.344714 8447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:53.345413 8447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [term 1 LEADER]: Becoming Leader. State: Replica: 11a79221473748f1ac794d89a6ec7379, State: Running, Role: LEADER
I20260809 06:30:53.345999 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:53.335967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:52953 (local address 127.0.0.1:39719)
0809 06:30:53.336150 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:53.336156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:53.339215 (+ 3059us) server_negotiation.cc:407] Connection header received
0809 06:30:53.339389 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:53.339392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:53.339449 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:53.339531 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:53.340244 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:53.341066 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:53.342940 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:53.343408 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:53.343537 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:53.344793 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:53.344800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:53.344804 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:53.344826 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:53.344900 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:53.344925 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:53.345058 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:53.345409 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 06:30:53.345491 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 06:30:53.346050 8447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a79221473748f1ac794d89a6ec7379" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39719 } }
I20260809 06:30:53.348578 8429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:53.349503 8450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11a79221473748f1ac794d89a6ec7379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a79221473748f1ac794d89a6ec7379" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39719 } } }
I20260809 06:30:53.349917 8450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:53.349725 8449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11a79221473748f1ac794d89a6ec7379. Latest consensus state: current_term: 1 leader_uuid: "11a79221473748f1ac794d89a6ec7379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a79221473748f1ac794d89a6ec7379" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39719 } } }
I20260809 06:30:53.350117 8449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379 [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:53.359251 8456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:53.364332 8456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:53.372917 8456 catalog_manager.cc:1347] Generated new cluster ID: 0d0e1f80a4514b45ae84a5a8eb35c704
I20260809 06:30:53.373044 8456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:53.385598 8456 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:53.387336 8456 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:53.394966 8456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11a79221473748f1ac794d89a6ec7379: Generated new TSK 0
I20260809 06:30:53.396013 8456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:53.407146 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8310
W20260809 06:30:53.408340 8420 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.116:36329
W20260809 06:30:53.409088 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8310: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:30:53.409245 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 8310
W20260809 06:30:53.431573 8278 connection.cc:535] client connection to 127.0.0.1:39719 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39719): Connection reset by peer (error 104)
2026-08-09T06:30:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2180 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:53 dist-test-slave-mjjr krb5kdc[8480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:53 dist-test-slave-mjjr krb5kdc[8480](info): set up 2 sockets
Aug 09 06:30:53 dist-test-slave-mjjr krb5kdc[8480](info): commencing operation
krb5kdc: starting...
W20260809 06:30:54.720345 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.001s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:54 dist-test-slave-mjjr krb5kdc[8480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 06:30:55.028191 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44851
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:55.368149 8503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:55.368425 8503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:55.368518 8503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:55.375618 8503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:55.375756 8503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:55.375801 8503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:55.375862 8503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:55.375926 8503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:55.385684 8503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44851
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:55.386899 8503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:55.388180 8503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:55.400012 8509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:55.399966 8512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:55.400081 8510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:55.401019 8503 server_base.cc:1034] running on GCE node
I20260809 06:30:55.401769 8503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:55.403139 8503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:55.404369 8503 hybrid_clock.cc:648] HybridClock initialized: now 1786257055404325 us; error 54 us; skew 500 ppm
I20260809 06:30:55.404740 8503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:55 dist-test-slave-mjjr krb5kdc[8480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257055, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:55.408008 8503 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 06:30:55.409550 8503 webserver.cc:460] Webserver started at http://127.0.0.1:36977/ using document root <none> and password file <none>
I20260809 06:30:55.410228 8503 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:55.410380 8503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:55.410722 8503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:55.413846 8503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "2809de2b060d4783a85421e73e087d2f"
format_stamp: "Formatted at 2026-08-09 06:30:55 on dist-test-slave-mjjr"
server_key: "b00726b77d75f688dfe3037719343eac"
server_key_iv: "5cddd3dbc28921903ff6c263086d7707"
server_key_version: "encryptionkey@0"
I20260809 06:30:55.414633 8503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "2809de2b060d4783a85421e73e087d2f"
format_stamp: "Formatted at 2026-08-09 06:30:55 on dist-test-slave-mjjr"
server_key: "b00726b77d75f688dfe3037719343eac"
server_key_iv: "5cddd3dbc28921903ff6c263086d7707"
server_key_version: "encryptionkey@0"
I20260809 06:30:55.419911 8503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 06:30:55.423185 8530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:55.424466 8503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:55.424656 8503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "2809de2b060d4783a85421e73e087d2f"
format_stamp: "Formatted at 2026-08-09 06:30:55 on dist-test-slave-mjjr"
server_key: "b00726b77d75f688dfe3037719343eac"
server_key_iv: "5cddd3dbc28921903ff6c263086d7707"
server_key_version: "encryptionkey@0"
I20260809 06:30:55.424851 8503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:55.430959 8503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:55.434743 8503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:55.435037 8503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:55.454025 8503 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44851
I20260809 06:30:55.454077 8617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44851 every 8 connection(s)
I20260809 06:30:55.455520 8503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:55.457150 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 8503
I20260809 06:30:55.457330 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:55.457862 8716 external_mini_cluster.cc:1428] Setting key 9a2d0c9d575fdca2f5c9295d331e1486
I20260809 06:30:55.460141 8619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:55.465448 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:55.459733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:39187 (local address 127.0.0.1:44851)
0809 06:30:55.460522 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:55.460534 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:55.460574 (+ 40us) server_negotiation.cc:407] Connection header received
0809 06:30:55.462188 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:55.462208 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:55.462528 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:55.462846 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:55.463939 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:55.463945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:55.463956 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:55.463971 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:55.464049 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:55.464069 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:55.464235 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:55.464360 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 06:30:55.464568 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":354,"threads_started":1}
I20260809 06:30:55.468990 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:55.471985 8619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f: Bootstrap starting.
I20260809 06:30:55.475740 8619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:55.475822 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:55.473398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:37179 (local address 127.0.0.1:44851)
0809 06:30:55.473602 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:55.473607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:55.473952 (+ 345us) server_negotiation.cc:407] Connection header received
0809 06:30:55.474116 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:55.474119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:55.474160 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:55.474237 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:55.475154 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:55.475157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:55.475160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:55.475174 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:55.475209 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:55.475223 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:55.475308 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:55.475453 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 06:30:55.475497 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 06:30:55.476881 8619 log.cc:824] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:55.479420 8619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f: No bootstrap required, opened a new log
I20260809 06:30:55.486593 8619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2809de2b060d4783a85421e73e087d2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44851 } }
I20260809 06:30:55.486954 8619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:55.487071 8619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2809de2b060d4783a85421e73e087d2f, State: Initialized, Role: FOLLOWER
I20260809 06:30:55.487612 8619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2809de2b060d4783a85421e73e087d2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44851 } }
I20260809 06:30:55.487890 8619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:55.488045 8619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:55.488245 8619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:55.490470 8619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2809de2b060d4783a85421e73e087d2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44851 } }
I20260809 06:30:55.491003 8619 leader_election.cc:302] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2809de2b060d4783a85421e73e087d2f; no voters:
I20260809 06:30:55.491621 8619 leader_election.cc:288] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:55.491778 8631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:55.492519 8631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [term 1 LEADER]: Becoming Leader. State: Replica: 2809de2b060d4783a85421e73e087d2f, State: Running, Role: LEADER
I20260809 06:30:55.493371 8631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2809de2b060d4783a85421e73e087d2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44851 } }
I20260809 06:30:55.493922 8619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:55.496271 8633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2809de2b060d4783a85421e73e087d2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2809de2b060d4783a85421e73e087d2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44851 } } }
I20260809 06:30:55.496886 8633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:55.496441 8634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2809de2b060d4783a85421e73e087d2f. Latest consensus state: current_term: 1 leader_uuid: "2809de2b060d4783a85421e73e087d2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2809de2b060d4783a85421e73e087d2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44851 } } }
I20260809 06:30:55.497468 8634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:55.498710 8645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:55.503964 8645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:55.513227 8645 catalog_manager.cc:1347] Generated new cluster ID: 561fc515628a4e3ca95e5dc966bb50b9
I20260809 06:30:55.513362 8645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:55.532125 8645 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:55.532570 8645 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:55.543102 8645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2809de2b060d4783a85421e73e087d2f: Generated new TSK 0
I20260809 06:30:55.543991 8645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:55.557600 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8503
W20260809 06:30:55.558845 8611 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.116:39187
W20260809 06:30:55.559569 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8503: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:30:55.559769 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 8503
2026-08-09T06:30:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2153 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:55 dist-test-slave-mjjr krb5kdc[8670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:55 dist-test-slave-mjjr krb5kdc[8670](info): set up 2 sockets
Aug 09 06:30:55 dist-test-slave-mjjr krb5kdc[8670](info): commencing operation
krb5kdc: starting...
W20260809 06:30:56.901060 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:57 dist-test-slave-mjjr krb5kdc[8670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:30:57.212257 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:42479
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:42479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:57.550722 8691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:57.551174 8691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:57.551334 8691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:57.559156 8691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:57.559265 8691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:57.559314 8691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:57.559377 8691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:57.559422 8691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:57.569425 8691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:42479
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:42479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:57.570636 8691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:57.571975 8691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:57.583760 8699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:57.583678 8700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:57.583734 8705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:57.584494 8691 server_base.cc:1034] running on GCE node
I20260809 06:30:57.585220 8691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:57.586558 8691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:57.587747 8691 hybrid_clock.cc:648] HybridClock initialized: now 1786257057587706 us; error 32 us; skew 500 ppm
I20260809 06:30:57.588121 8691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:57 dist-test-slave-mjjr krb5kdc[8670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257057, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:57.595595 8691 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:30:57.597281 8691 webserver.cc:460] Webserver started at http://127.8.131.62:37413/ using document root <none> and password file <none>
I20260809 06:30:57.597999 8691 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:57.598137 8691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:57.598472 8691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:57.601497 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "0c82a30cd4e94284adbf57217e8339ae"
format_stamp: "Formatted at 2026-08-09 06:30:57 on dist-test-slave-mjjr"
server_key: "43660ff117ae4d835fb0dc57f77fd1a8"
server_key_iv: "a8a86de23a6e3bf73b3e27e2170dc229"
server_key_version: "encryptionkey@0"
I20260809 06:30:57.602268 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "0c82a30cd4e94284adbf57217e8339ae"
format_stamp: "Formatted at 2026-08-09 06:30:57 on dist-test-slave-mjjr"
server_key: "43660ff117ae4d835fb0dc57f77fd1a8"
server_key_iv: "a8a86de23a6e3bf73b3e27e2170dc229"
server_key_version: "encryptionkey@0"
I20260809 06:30:57.607474 8691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 06:30:57.610931 8724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:57.612206 8691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:57.612403 8691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "0c82a30cd4e94284adbf57217e8339ae"
format_stamp: "Formatted at 2026-08-09 06:30:57 on dist-test-slave-mjjr"
server_key: "43660ff117ae4d835fb0dc57f77fd1a8"
server_key_iv: "a8a86de23a6e3bf73b3e27e2170dc229"
server_key_version: "encryptionkey@0"
I20260809 06:30:57.612607 8691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:57.641242 8691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:57.645192 8691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:57.645496 8691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:57.664598 8691 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:42479
I20260809 06:30:57.664633 8799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:42479 every 8 connection(s)
I20260809 06:30:57.666003 8691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:57.669747 8800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:57.671361 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 8691
I20260809 06:30:57.671551 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:57.672030 8716 external_mini_cluster.cc:1428] Setting key 694c25db3d8467a9759af67ddd55fb82
I20260809 06:30:57.684221 8800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae: Bootstrap starting.
Aug 09 06:30:57 dist-test-slave-mjjr krb5kdc[8670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:30:57.687386 8800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:57.688680 8800 log.cc:824] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:57.691416 8800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae: No bootstrap required, opened a new log
I20260809 06:30:57.695462 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:57.673926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57517 (local address 127.8.131.62:42479)
0809 06:30:57.674738 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:57.674752 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:57.674804 (+ 52us) server_negotiation.cc:407] Connection header received
0809 06:30:57.676562 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:57.676592 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:57.677087 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:57.677521 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:57.678840 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:57.680880 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:57.682135 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:57.683168 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:57.683337 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:57.687739 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:57.687749 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:57.687766 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:57.687810 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:57.690901 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:57.691691 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:57.691699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:57.691706 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:57.691921 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:57.692402 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:57.692407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:57.692410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:57.693071 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:57.693419 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:57.693750 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 06:30:57.694136 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":324,"threads_started":1}
I20260809 06:30:57.699231 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:57.700635 8800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c82a30cd4e94284adbf57217e8339ae" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42479 } }
I20260809 06:30:57.700990 8800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:57.701088 8800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c82a30cd4e94284adbf57217e8339ae, State: Initialized, Role: FOLLOWER
I20260809 06:30:57.701699 8800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c82a30cd4e94284adbf57217e8339ae" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42479 } }
I20260809 06:30:57.701952 8800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:57.702117 8800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:57.702294 8800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:57.704752 8800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c82a30cd4e94284adbf57217e8339ae" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42479 } }
I20260809 06:30:57.705233 8800 leader_election.cc:302] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c82a30cd4e94284adbf57217e8339ae; no voters:
I20260809 06:30:57.705922 8800 leader_election.cc:288] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:57.706017 8815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:57.707312 8815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [term 1 LEADER]: Becoming Leader. State: Replica: 0c82a30cd4e94284adbf57217e8339ae, State: Running, Role: LEADER
I20260809 06:30:57.707935 8815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c82a30cd4e94284adbf57217e8339ae" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42479 } }
I20260809 06:30:57.708385 8800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:57.710971 8816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c82a30cd4e94284adbf57217e8339ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c82a30cd4e94284adbf57217e8339ae" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42479 } } }
I20260809 06:30:57.711426 8816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:57.713223 8819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c82a30cd4e94284adbf57217e8339ae. Latest consensus state: current_term: 1 leader_uuid: "0c82a30cd4e94284adbf57217e8339ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c82a30cd4e94284adbf57217e8339ae" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42479 } } }
I20260809 06:30:57.713600 8819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:57.713769 8825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:57.720086 8825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:57.725261 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:57.703874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40677 (local address 127.8.131.62:42479)
0809 06:30:57.704117 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:57.704125 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:57.705988 (+ 1863us) server_negotiation.cc:407] Connection header received
0809 06:30:57.706287 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:57.706294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:57.706393 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:57.706528 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:57.708551 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:57.709370 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:57.711248 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:57.712028 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:57.712148 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:57.717055 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:57.717064 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:57.717068 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:30:57.717111 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:57.721222 (+ 4111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:57.722293 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:57.722300 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:57.722304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:57.722440 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:30:57.723054 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:30:57.723064 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:30:57.723068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:30:57.723728 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:57.724030 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:57.724709 (+ 679us) server_negotiation.cc:299] Negotiation successful
0809 06:30:57.724905 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:30:57.731243 8825 catalog_manager.cc:1347] Generated new cluster ID: 8182a0cc3eb948c4887079eb7772d158
I20260809 06:30:57.731350 8825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:30:57.744228 8825 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:30:57.745561 8825 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:30:57.755308 8825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c82a30cd4e94284adbf57217e8339ae: Generated new TSK 0
I20260809 06:30:57.756155 8825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:30:57.763746 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8691
I20260809 06:30:57.764925 8794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:30:57.856597 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 8691
W20260809 06:30:57.883343 8665 connection.cc:535] client connection to 127.8.131.62:42479 recv error: Network error: failed to read from TLS socket (remote: 127.8.131.62:42479): Connection reset by peer (error 104)
2026-08-09T06:30:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:30:57 dist-test-slave-mjjr krb5kdc[8848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:30:57 dist-test-slave-mjjr krb5kdc[8848](info): set up 2 sockets
Aug 09 06:30:57 dist-test-slave-mjjr krb5kdc[8848](info): commencing operation
krb5kdc: starting...
W20260809 06:30:59.179364 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.006s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:30:59 dist-test-slave-mjjr krb5kdc[8848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:30:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:30:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:30:59.472841 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41873
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:41747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:41873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:30:59.813825 8871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:30:59.814110 8871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:30:59.814517 8871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:30:59.821954 8871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:30:59.822088 8871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:30:59.822191 8871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:30:59.822314 8871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:30:59.822394 8871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:30:59.832513 8871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:41747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:41873
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:41873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:30:59.833861 8871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:30:59.835201 8871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:30:59.847133 8880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:59.847205 8877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:30:59.847220 8878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:30:59.853121 8871 server_base.cc:1034] running on GCE node
I20260809 06:30:59.853921 8871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:30:59.855329 8871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:30:59.856563 8871 hybrid_clock.cc:648] HybridClock initialized: now 1786257059856523 us; error 52 us; skew 500 ppm
I20260809 06:30:59.856926 8871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:30:59 dist-test-slave-mjjr krb5kdc[8848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257059, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:30:59.860039 8871 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:30:59.861579 8871 webserver.cc:460] Webserver started at http://127.8.131.62:34175/ using document root <none> and password file <none>
I20260809 06:30:59.862284 8871 fs_manager.cc:359] Metadata directory not provided
I20260809 06:30:59.862437 8871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:30:59.862746 8871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:30:59.866875 8871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "b462dd42c5714691b353f4054e0bd50d"
format_stamp: "Formatted at 2026-08-09 06:30:59 on dist-test-slave-mjjr"
server_key: "8fd7908854e63b0354429d3dc5b26483"
server_key_iv: "abaad409640cbcdbe41161f07a246bfc"
server_key_version: "encryptionkey@0"
I20260809 06:30:59.867934 8871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "b462dd42c5714691b353f4054e0bd50d"
format_stamp: "Formatted at 2026-08-09 06:30:59 on dist-test-slave-mjjr"
server_key: "8fd7908854e63b0354429d3dc5b26483"
server_key_iv: "abaad409640cbcdbe41161f07a246bfc"
server_key_version: "encryptionkey@0"
I20260809 06:30:59.876416 8871 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260809 06:30:59.880622 8891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:30:59.881985 8871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:30:59.882134 8871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "b462dd42c5714691b353f4054e0bd50d"
format_stamp: "Formatted at 2026-08-09 06:30:59 on dist-test-slave-mjjr"
server_key: "8fd7908854e63b0354429d3dc5b26483"
server_key_iv: "abaad409640cbcdbe41161f07a246bfc"
server_key_version: "encryptionkey@0"
I20260809 06:30:59.882303 8871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:30:59.907121 8871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:30:59.911092 8871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:30:59.911334 8871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:30:59.930197 8871 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:41873
I20260809 06:30:59.930238 8972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:41873 every 8 connection(s)
I20260809 06:30:59.931545 8871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:30:59.935216 8973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:30:59.935528 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 8871
I20260809 06:30:59.935725 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:30:59.936197 8716 external_mini_cluster.cc:1428] Setting key a5fdbaa27ecc11297e68b717ef984ea9
I20260809 06:30:59.948125 8973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d: Bootstrap starting.
I20260809 06:30:59.949110 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:59.938008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56281 (local address 127.8.131.62:41873)
0809 06:30:59.938807 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:59.938818 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:59.938856 (+ 38us) server_negotiation.cc:407] Connection header received
0809 06:30:59.940517 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:59.940538 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:59.940951 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:59.941357 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:59.941909 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:59.943219 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:59.944603 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:59.945309 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:59.945547 (+ 238us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:59.946865 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:59.946873 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:59.946885 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:59.947400 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:59.947481 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:59.947506 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:59.947700 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:59.947979 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 06:30:59.948153 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":389,"threads_started":1}
I20260809 06:30:59.951335 8973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d: Neither blocks nor log segments found. Creating new log.
I20260809 06:30:59.952584 8973 log.cc:824] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d: Log is configured to *not* fsync() on all Append() calls
I20260809 06:30:59.952836 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:30:59.955729 8973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d: No bootstrap required, opened a new log
I20260809 06:30:59.964708 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:30:59.958064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45417 (local address 127.8.131.62:41873)
0809 06:30:59.958241 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 06:30:59.958247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:30:59.958612 (+ 365us) server_negotiation.cc:407] Connection header received
0809 06:30:59.958758 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:30:59.958764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:30:59.958835 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:30:59.958963 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:30:59.959431 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:59.960609 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:59.961923 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:30:59.962622 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:30:59.962759 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:30:59.963865 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:30:59.963871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:30:59.963876 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:30:59.963897 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:30:59.963965 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:30:59.963995 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:30:59.964104 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 06:30:59.964289 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 06:30:59.964342 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:30:59.965725 8973 raft_consensus.cc:348] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b462dd42c5714691b353f4054e0bd50d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41873 } }
I20260809 06:30:59.966147 8973 raft_consensus.cc:374] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:30:59.966261 8973 raft_consensus.cc:729] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b462dd42c5714691b353f4054e0bd50d, State: Initialized, Role: FOLLOWER
I20260809 06:30:59.966823 8973 consensus_queue.cc:260] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b462dd42c5714691b353f4054e0bd50d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41873 } }
I20260809 06:30:59.967067 8973 raft_consensus.cc:388] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:30:59.967216 8973 raft_consensus.cc:482] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:30:59.967406 8973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:30:59.969578 8973 raft_consensus.cc:504] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b462dd42c5714691b353f4054e0bd50d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41873 } }
I20260809 06:30:59.970098 8973 leader_election.cc:302] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b462dd42c5714691b353f4054e0bd50d; no voters:
I20260809 06:30:59.970649 8973 leader_election.cc:288] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:30:59.970778 8988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:30:59.971485 8988 raft_consensus.cc:686] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [term 1 LEADER]: Becoming Leader. State: Replica: b462dd42c5714691b353f4054e0bd50d, State: Running, Role: LEADER
I20260809 06:30:59.972170 8988 consensus_queue.cc:237] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b462dd42c5714691b353f4054e0bd50d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41873 } }
I20260809 06:30:59.972954 8973 sys_catalog.cc:564] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:30:59.975441 8990 sys_catalog.cc:455] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b462dd42c5714691b353f4054e0bd50d. Latest consensus state: current_term: 1 leader_uuid: "b462dd42c5714691b353f4054e0bd50d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b462dd42c5714691b353f4054e0bd50d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41873 } } }
I20260809 06:30:59.975597 8989 sys_catalog.cc:455] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b462dd42c5714691b353f4054e0bd50d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b462dd42c5714691b353f4054e0bd50d" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 41873 } } }
I20260809 06:30:59.975905 8990 sys_catalog.cc:458] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:59.975932 8989 sys_catalog.cc:458] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d [sys.catalog]: This master's current role is: LEADER
I20260809 06:30:59.978807 8999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:30:59.984282 8999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:30:59.993407 8999 catalog_manager.cc:1347] Generated new cluster ID: ada3fe57b57a4773ac3bbafcbdd2f243
I20260809 06:30:59.993551 8999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:31:00.004752 8999 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:31:00.006135 8999 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:31:00.014463 8999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b462dd42c5714691b353f4054e0bd50d: Generated new TSK 0
I20260809 06:31:00.015290 8999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:31:00.023662 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8871
W20260809 06:31:00.024803 8964 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56281
W20260809 06:31:00.025549 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 8871: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:31:00.025697 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 8871
W20260809 06:31:00.053041 8844 connection.cc:535] client connection to 127.8.131.62:41873 recv error: Network error: failed to read from TLS socket (remote: 127.8.131.62:41873): Connection reset by peer (error 104)
2026-08-09T06:31:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2169 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:31:00 dist-test-slave-mjjr krb5kdc[9020](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:31:00 dist-test-slave-mjjr krb5kdc[9020](info): set up 2 sockets
Aug 09 06:31:00 dist-test-slave-mjjr krb5kdc[9020](info): commencing operation
krb5kdc: starting...
W20260809 06:31:01.348932 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:31:01 dist-test-slave-mjjr krb5kdc[9020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:31:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:31:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:31:01.685029 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:40735
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:33005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:40735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:31:02.028816 9038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:02.029109 9038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:02.029299 9038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:31:02.036636 9038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:02.036741 9038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:02.036882 9038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:02.037026 9038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:02.037099 9038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:02.047132 9038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:33005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:40735
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:40735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:02.048442 9038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:02.049760 9038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:02.061899 9047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:02.061851 9051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:02.061973 9048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:02.063198 9038 server_base.cc:1034] running on GCE node
I20260809 06:31:02.064095 9038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:02.065474 9038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:02.066720 9038 hybrid_clock.cc:648] HybridClock initialized: now 1786257062066679 us; error 48 us; skew 500 ppm
I20260809 06:31:02.067099 9038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:02 dist-test-slave-mjjr krb5kdc[9020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257062, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:02.070247 9038 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:31:02.071808 9038 webserver.cc:460] Webserver started at http://127.8.131.62:33435/ using document root <none> and password file <none>
I20260809 06:31:02.072481 9038 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:02.072634 9038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:02.072947 9038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:02.075989 9038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "c5388e2680cc4bd4806d4fcfad177187"
format_stamp: "Formatted at 2026-08-09 06:31:02 on dist-test-slave-mjjr"
server_key: "9c8332a95e200ce22b50e6875343eee0"
server_key_iv: "41b707ad7df7b4774a2e5c5ff72c3c78"
server_key_version: "encryptionkey@0"
I20260809 06:31:02.076750 9038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "c5388e2680cc4bd4806d4fcfad177187"
format_stamp: "Formatted at 2026-08-09 06:31:02 on dist-test-slave-mjjr"
server_key: "9c8332a95e200ce22b50e6875343eee0"
server_key_iv: "41b707ad7df7b4774a2e5c5ff72c3c78"
server_key_version: "encryptionkey@0"
I20260809 06:31:02.081846 9038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:31:02.085183 9064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:02.086500 9038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:31:02.086694 9038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "c5388e2680cc4bd4806d4fcfad177187"
format_stamp: "Formatted at 2026-08-09 06:31:02 on dist-test-slave-mjjr"
server_key: "9c8332a95e200ce22b50e6875343eee0"
server_key_iv: "41b707ad7df7b4774a2e5c5ff72c3c78"
server_key_version: "encryptionkey@0"
I20260809 06:31:02.086896 9038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:02.092602 9038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:02.096431 9038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:02.096731 9038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:02.115177 9038 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:40735
I20260809 06:31:02.115224 9143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:40735 every 8 connection(s)
I20260809 06:31:02.116681 9038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:31:02.120657 9146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:31:02.124286 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9038
I20260809 06:31:02.124490 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:31:02.125114 8716 external_mini_cluster.cc:1428] Setting key b6a91883740a26c8017accad7969c4ca
I20260809 06:31:02.132737 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:02.126971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48339 (local address 127.8.131.62:40735)
0809 06:31:02.127795 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:02.127807 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:02.127847 (+ 40us) server_negotiation.cc:407] Connection header received
0809 06:31:02.129449 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:02.129469 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:02.129782 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:02.130092 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:02.131207 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:02.131212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:02.131224 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:31:02.131238 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:02.131307 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:31:02.131328 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:02.131509 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:02.131618 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 06:31:02.131875 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":380,"threads_started":1}
I20260809 06:31:02.133651 9146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187: Bootstrap starting.
I20260809 06:31:02.136782 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:31:02.137806 9146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:02.139159 9146 log.cc:824] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187: Log is configured to *not* fsync() on all Append() calls
I20260809 06:31:02.142103 9146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187: No bootstrap required, opened a new log
I20260809 06:31:02.145308 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:02.142417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38435 (local address 127.8.131.62:40735)
0809 06:31:02.142620 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:02.142632 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:02.143164 (+ 532us) server_negotiation.cc:407] Connection header received
0809 06:31:02.143322 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:02.143328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:02.143388 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:02.143500 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:02.144557 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:02.144563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:02.144567 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:31:02.144586 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:02.144643 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:31:02.144663 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:02.144778 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:02.144888 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 06:31:02.144958 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:31:02.150914 9146 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5388e2680cc4bd4806d4fcfad177187" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40735 } }
I20260809 06:31:02.151258 9146 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:31:02.151331 9146 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5388e2680cc4bd4806d4fcfad177187, State: Initialized, Role: FOLLOWER
I20260809 06:31:02.151918 9146 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5388e2680cc4bd4806d4fcfad177187" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40735 } }
I20260809 06:31:02.152125 9146 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:31:02.152225 9146 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:31:02.152343 9146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:02.154675 9146 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5388e2680cc4bd4806d4fcfad177187" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40735 } }
I20260809 06:31:02.155215 9146 leader_election.cc:302] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5388e2680cc4bd4806d4fcfad177187; no voters:
I20260809 06:31:02.155825 9146 leader_election.cc:288] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:31:02.155963 9164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:31:02.156690 9164 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [term 1 LEADER]: Becoming Leader. State: Replica: c5388e2680cc4bd4806d4fcfad177187, State: Running, Role: LEADER
I20260809 06:31:02.157231 9164 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5388e2680cc4bd4806d4fcfad177187" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40735 } }
I20260809 06:31:02.158035 9146 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:31:02.160342 9167 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5388e2680cc4bd4806d4fcfad177187. Latest consensus state: current_term: 1 leader_uuid: "c5388e2680cc4bd4806d4fcfad177187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5388e2680cc4bd4806d4fcfad177187" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40735 } } }
I20260809 06:31:02.160742 9167 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:02.161185 9165 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5388e2680cc4bd4806d4fcfad177187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5388e2680cc4bd4806d4fcfad177187" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40735 } } }
I20260809 06:31:02.161487 9165 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:02.163373 9173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:31:02.168557 9173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:31:02.177001 9173 catalog_manager.cc:1347] Generated new cluster ID: 5efdb74a46db41db8894463175e5759a
I20260809 06:31:02.177106 9173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:31:02.191416 9173 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:31:02.191845 9173 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:31:02.204265 9173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5388e2680cc4bd4806d4fcfad177187: Generated new TSK 0
I20260809 06:31:02.205183 9173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:31:02.219461 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9038
W20260809 06:31:02.220942 9139 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48339
W20260809 06:31:02.221644 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9038: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:31:02.221787 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9038
2026-08-09T06:31:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2195 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:31:02 dist-test-slave-mjjr krb5kdc[9197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:31:02 dist-test-slave-mjjr krb5kdc[9197](info): set up 2 sockets
Aug 09 06:31:02 dist-test-slave-mjjr krb5kdc[9197](info): commencing operation
krb5kdc: starting...
W20260809 06:31:03.548908 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:31:03 dist-test-slave-mjjr krb5kdc[9197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:31:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:31:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:31:03.867832 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:42195
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:35751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:42195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:31:04.219213 9220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:04.219502 9220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:04.219594 9220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:31:04.226622 9220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:04.226719 9220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:04.226765 9220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:04.226826 9220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:04.226863 9220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:04.236299 9220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:35751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:42195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:42195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:04.237430 9220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:04.238674 9220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:04.250639 9227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:04.250586 9229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:04.251017 9231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:04.251670 9220 server_base.cc:1034] running on GCE node
I20260809 06:31:04.252454 9220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:04.253839 9220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:04.255074 9220 hybrid_clock.cc:648] HybridClock initialized: now 1786257064255017 us; error 40 us; skew 500 ppm
I20260809 06:31:04.255422 9220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:04 dist-test-slave-mjjr krb5kdc[9197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257064, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:04.261746 9220 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:31:04.263338 9220 webserver.cc:460] Webserver started at http://127.8.131.62:33805/ using document root <none> and password file <none>
I20260809 06:31:04.264098 9220 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:04.264247 9220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:04.264561 9220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:04.267537 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8"
format_stamp: "Formatted at 2026-08-09 06:31:04 on dist-test-slave-mjjr"
server_key: "144ba6fcd2c0177f1a43eb69044f1485"
server_key_iv: "a12064388890efd460736620b0c70ca7"
server_key_version: "encryptionkey@0"
I20260809 06:31:04.268359 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8"
format_stamp: "Formatted at 2026-08-09 06:31:04 on dist-test-slave-mjjr"
server_key: "144ba6fcd2c0177f1a43eb69044f1485"
server_key_iv: "a12064388890efd460736620b0c70ca7"
server_key_version: "encryptionkey@0"
I20260809 06:31:04.273563 9220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 06:31:04.277055 9244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:04.278296 9220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:31:04.278507 9220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8"
format_stamp: "Formatted at 2026-08-09 06:31:04 on dist-test-slave-mjjr"
server_key: "144ba6fcd2c0177f1a43eb69044f1485"
server_key_iv: "a12064388890efd460736620b0c70ca7"
server_key_version: "encryptionkey@0"
I20260809 06:31:04.278728 9220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:04.302950 9220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:04.307344 9220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:04.307600 9220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:04.326478 9220 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:42195
I20260809 06:31:04.326521 9334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:42195 every 8 connection(s)
I20260809 06:31:04.327951 9220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:31:04.330000 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9220
I20260809 06:31:04.330186 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:31:04.330657 8716 external_mini_cluster.cc:1428] Setting key 3e618cd6f8ea3d553069c1432e653eaf
I20260809 06:31:04.332268 9335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 06:31:04 dist-test-slave-mjjr krb5kdc[9197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:31:04.346297 9335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8: Bootstrap starting.
I20260809 06:31:04.349524 9335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:04.350643 9335 log.cc:824] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8: Log is configured to *not* fsync() on all Append() calls
I20260809 06:31:04.353962 9335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8: No bootstrap required, opened a new log
I20260809 06:31:04.354188 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:04.332634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36125 (local address 127.8.131.62:42195)
0809 06:31:04.333460 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:04.333472 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:04.333511 (+ 39us) server_negotiation.cc:407] Connection header received
0809 06:31:04.335083 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:04.335102 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:04.335519 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:04.335911 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:04.337782 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:04.339110 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:04.340858 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:04.341502 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:04.341631 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:04.346495 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:04.346512 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:04.346527 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:04.346564 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:04.349710 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:04.350469 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:04.350477 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:04.350495 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:04.350722 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:04.351245 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:04.351252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:04.351258 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:04.352081 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:04.352341 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:04.352799 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 06:31:04.353144 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":368,"threads_started":1}
I20260809 06:31:04.357846 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:31:04.361865 9335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42195 } }
I20260809 06:31:04.362326 9335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:31:04.362439 9335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1eb7a5df094bacb0c0442e3bccf2e8, State: Initialized, Role: FOLLOWER
I20260809 06:31:04.363113 9335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42195 } }
I20260809 06:31:04.363409 9335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:31:04.363577 9335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:31:04.363827 9335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:04.366780 9335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42195 } }
I20260809 06:31:04.367326 9335 leader_election.cc:302] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d1eb7a5df094bacb0c0442e3bccf2e8; no voters:
I20260809 06:31:04.368032 9335 leader_election.cc:288] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:31:04.368157 9354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:31:04.370003 9354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1d1eb7a5df094bacb0c0442e3bccf2e8, State: Running, Role: LEADER
I20260809 06:31:04.370258 9335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:31:04.370573 9354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42195 } }
I20260809 06:31:04.374030 9359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42195 } } }
I20260809 06:31:04.374480 9359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:04.374986 9360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d1eb7a5df094bacb0c0442e3bccf2e8. Latest consensus state: current_term: 1 leader_uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1eb7a5df094bacb0c0442e3bccf2e8" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 42195 } } }
I20260809 06:31:04.375278 9360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:04.381706 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:04.362218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42339 (local address 127.8.131.62:42195)
0809 06:31:04.362481 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:04.362488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:04.363008 (+ 520us) server_negotiation.cc:407] Connection header received
0809 06:31:04.363176 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:04.363179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:04.363234 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:04.363308 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:04.365262 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:04.366011 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:04.367832 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:04.368617 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:04.368787 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:04.372236 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:04.372265 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:04.372272 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:04.372319 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:04.376917 (+ 4598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:04.377948 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:04.377956 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:04.377961 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:04.378160 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:04.378705 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:04.378712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:04.378718 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:04.379168 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:04.379507 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:04.380866 (+ 1359us) server_negotiation.cc:299] Negotiation successful
0809 06:31:04.381144 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260809 06:31:04.384686 9372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:31:04.390163 9372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:31:04.398906 9372 catalog_manager.cc:1347] Generated new cluster ID: a85812d17f924b80822860ae609656fd
I20260809 06:31:04.399024 9372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:31:04.431066 9372 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:31:04.432502 9372 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:31:04.443099 9372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d1eb7a5df094bacb0c0442e3bccf2e8: Generated new TSK 0
I20260809 06:31:04.443951 9372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:31:04.457315 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9220
I20260809 06:31:04.458518 9324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:31:04.551348 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9220
2026-08-09T06:31:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:31:04 dist-test-slave-mjjr krb5kdc[9390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:31:04 dist-test-slave-mjjr krb5kdc[9390](info): set up 2 sockets
Aug 09 06:31:04 dist-test-slave-mjjr krb5kdc[9390](info): commencing operation
krb5kdc: starting...
W20260809 06:31:05.880825 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:31:06 dist-test-slave-mjjr krb5kdc[9390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:31:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:31:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:31:06.187777 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:46149
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:33745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:46149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:31:06.527803 9410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:06.528105 9410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:06.528200 9410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:31:06.535392 9410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:06.535509 9410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:06.535557 9410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:06.535620 9410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:06.535683 9410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:06.545459 9410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:33745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:46149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:46149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:06.546753 9410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:06.548022 9410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:06.559876 9419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:06.559991 9418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:06.559962 9426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:06.560921 9410 server_base.cc:1034] running on GCE node
I20260809 06:31:06.561700 9410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:06.563084 9410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:06.564311 9410 hybrid_clock.cc:648] HybridClock initialized: now 1786257066564262 us; error 60 us; skew 500 ppm
I20260809 06:31:06.564664 9410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:06 dist-test-slave-mjjr krb5kdc[9390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257066, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:06.567821 9410 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:31:06.569182 9410 webserver.cc:460] Webserver started at http://127.8.131.62:40017/ using document root <none> and password file <none>
I20260809 06:31:06.569810 9410 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:06.569909 9410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:06.570143 9410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:06.573102 9410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "f0be5749202f4b9bac0bc6c9fd1591d0"
format_stamp: "Formatted at 2026-08-09 06:31:06 on dist-test-slave-mjjr"
server_key: "47e1a32e1c90bd2d97391d4c3f5f2b3a"
server_key_iv: "5000c2170f82637c5bde5162b55a358a"
server_key_version: "encryptionkey@0"
I20260809 06:31:06.573805 9410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "f0be5749202f4b9bac0bc6c9fd1591d0"
format_stamp: "Formatted at 2026-08-09 06:31:06 on dist-test-slave-mjjr"
server_key: "47e1a32e1c90bd2d97391d4c3f5f2b3a"
server_key_iv: "5000c2170f82637c5bde5162b55a358a"
server_key_version: "encryptionkey@0"
I20260809 06:31:06.578841 9410 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 06:31:06.582165 9439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:06.583423 9410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 06:31:06.583578 9410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "f0be5749202f4b9bac0bc6c9fd1591d0"
format_stamp: "Formatted at 2026-08-09 06:31:06 on dist-test-slave-mjjr"
server_key: "47e1a32e1c90bd2d97391d4c3f5f2b3a"
server_key_iv: "5000c2170f82637c5bde5162b55a358a"
server_key_version: "encryptionkey@0"
I20260809 06:31:06.583820 9410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:06.601572 9410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:06.605631 9410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:06.605878 9410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:06.625234 9410 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:46149
I20260809 06:31:06.625277 9511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:46149 every 8 connection(s)
I20260809 06:31:06.626685 9410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:31:06.627166 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9410
I20260809 06:31:06.627360 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:31:06.628001 8716 external_mini_cluster.cc:1428] Setting key 6dcb890436ba9707bd13376615750110
I20260809 06:31:06.631167 9514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:31:06.640877 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:06.629837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56657 (local address 127.8.131.62:46149)
0809 06:31:06.630578 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:06.630590 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:06.630632 (+ 42us) server_negotiation.cc:407] Connection header received
0809 06:31:06.632315 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:06.632336 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:06.632700 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:06.633076 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:06.633642 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:06.634959 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:06.636382 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:06.637059 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:06.637262 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:06.638508 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:06.638516 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:06.638529 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:31:06.639108 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:06.639191 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:31:06.639225 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:06.639388 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:06.639619 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 06:31:06.639920 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":314,"threads_started":1}
I20260809 06:31:06.642195 9514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0: Bootstrap starting.
I20260809 06:31:06.644763 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:31:06.645696 9514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:06.646732 9514 log.cc:824] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0: Log is configured to *not* fsync() on all Append() calls
I20260809 06:31:06.649729 9514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0: No bootstrap required, opened a new log
I20260809 06:31:06.655929 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:06.649303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51815 (local address 127.8.131.62:46149)
0809 06:31:06.649488 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:06.649494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:06.649933 (+ 439us) server_negotiation.cc:407] Connection header received
0809 06:31:06.650106 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:06.650110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:06.650200 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:06.650331 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:06.650794 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:06.651958 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:06.653278 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:06.653973 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:06.654101 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:06.655127 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:06.655135 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:06.655145 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:31:06.655167 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:06.655211 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:31:06.655230 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:06.655326 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:06.655456 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 06:31:06.655561 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 06:31:06.658892 9514 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0be5749202f4b9bac0bc6c9fd1591d0" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 46149 } }
I20260809 06:31:06.659263 9514 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:31:06.659377 9514 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0be5749202f4b9bac0bc6c9fd1591d0, State: Initialized, Role: FOLLOWER
I20260809 06:31:06.660000 9514 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0be5749202f4b9bac0bc6c9fd1591d0" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 46149 } }
I20260809 06:31:06.660238 9514 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:31:06.660388 9514 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:31:06.660570 9514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:06.662714 9514 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0be5749202f4b9bac0bc6c9fd1591d0" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 46149 } }
I20260809 06:31:06.663311 9514 leader_election.cc:302] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0be5749202f4b9bac0bc6c9fd1591d0; no voters:
I20260809 06:31:06.663966 9514 leader_election.cc:288] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:31:06.664108 9525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:31:06.664794 9525 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [term 1 LEADER]: Becoming Leader. State: Replica: f0be5749202f4b9bac0bc6c9fd1591d0, State: Running, Role: LEADER
I20260809 06:31:06.665304 9525 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0be5749202f4b9bac0bc6c9fd1591d0" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 46149 } }
I20260809 06:31:06.666013 9514 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:31:06.668411 9527 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0be5749202f4b9bac0bc6c9fd1591d0. Latest consensus state: current_term: 1 leader_uuid: "f0be5749202f4b9bac0bc6c9fd1591d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0be5749202f4b9bac0bc6c9fd1591d0" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 46149 } } }
I20260809 06:31:06.668805 9527 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:06.668582 9526 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0be5749202f4b9bac0bc6c9fd1591d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0be5749202f4b9bac0bc6c9fd1591d0" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 46149 } } }
I20260809 06:31:06.668973 9526 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:06.669818 9533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:31:06.676218 9533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:31:06.685274 9533 catalog_manager.cc:1347] Generated new cluster ID: 246bd3f25b81469ba0d362520147ed15
I20260809 06:31:06.685413 9533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:31:06.707333 9533 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:31:06.709209 9533 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:31:06.718544 9533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0be5749202f4b9bac0bc6c9fd1591d0: Generated new TSK 0
I20260809 06:31:06.719511 9533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:31:06.726814 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9410
W20260809 06:31:06.727838 9507 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56657
W20260809 06:31:06.728525 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9410: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:31:06.728663 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9410
2026-08-09T06:31:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2178 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:31:06 dist-test-slave-mjjr krb5kdc[9560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:31:06 dist-test-slave-mjjr krb5kdc[9560](info): set up 2 sockets
Aug 09 06:31:06 dist-test-slave-mjjr krb5kdc[9560](info): commencing operation
krb5kdc: starting...
W20260809 06:31:08.042716 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:31:08 dist-test-slave-mjjr krb5kdc[9560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:31:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:31:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:31:08.352607 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:35245
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:37091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:35245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:31:08.692987 9586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:08.693324 9586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:08.693490 9586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:31:08.700726 9586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:08.700896 9586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:08.701001 9586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:08.701112 9586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:08.701189 9586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:08.711177 9586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:35245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:35245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:08.712538 9586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:08.713853 9586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:08.726058 9592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:08.725955 9597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:08.725992 9595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:08.726878 9586 server_base.cc:1034] running on GCE node
I20260809 06:31:08.727759 9586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:08.729112 9586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:08.730356 9586 hybrid_clock.cc:648] HybridClock initialized: now 1786257068730310 us; error 56 us; skew 500 ppm
I20260809 06:31:08.730724 9586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:08 dist-test-slave-mjjr krb5kdc[9560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257068, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:08.733852 9586 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:31:08.735311 9586 webserver.cc:460] Webserver started at http://127.8.131.62:44431/ using document root <none> and password file <none>
I20260809 06:31:08.736070 9586 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:08.736222 9586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:08.736539 9586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:08.739495 9586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "7578a6939f084e19a3108742dd750831"
format_stamp: "Formatted at 2026-08-09 06:31:08 on dist-test-slave-mjjr"
server_key: "ba595e9acd0ee2f5eec9f73534d39b2e"
server_key_iv: "b0bdf2680244fb64721a510496876f34"
server_key_version: "encryptionkey@0"
I20260809 06:31:08.740337 9586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "7578a6939f084e19a3108742dd750831"
format_stamp: "Formatted at 2026-08-09 06:31:08 on dist-test-slave-mjjr"
server_key: "ba595e9acd0ee2f5eec9f73534d39b2e"
server_key_iv: "b0bdf2680244fb64721a510496876f34"
server_key_version: "encryptionkey@0"
I20260809 06:31:08.745497 9586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:31:08.748903 9606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:08.750144 9586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 06:31:08.750321 9586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "7578a6939f084e19a3108742dd750831"
format_stamp: "Formatted at 2026-08-09 06:31:08 on dist-test-slave-mjjr"
server_key: "ba595e9acd0ee2f5eec9f73534d39b2e"
server_key_iv: "b0bdf2680244fb64721a510496876f34"
server_key_version: "encryptionkey@0"
I20260809 06:31:08.750526 9586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:08.756294 9586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:08.760193 9586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:08.760491 9586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:08.779316 9586 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:35245
I20260809 06:31:08.779364 9680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:35245 every 8 connection(s)
I20260809 06:31:08.780787 9586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:31:08.784305 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9586
I20260809 06:31:08.784504 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:31:08.784855 9682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:31:08.785122 8716 external_mini_cluster.cc:1428] Setting key 907374b0e724c8dfc4e3dd1f1ef9b104
I20260809 06:31:08.793081 9685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:08.786971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45303 (local address 127.8.131.62:35245)
0809 06:31:08.787839 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:08.787850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:08.787895 (+ 45us) server_negotiation.cc:407] Connection header received
0809 06:31:08.789656 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:08.789676 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:08.790004 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:08.790315 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:08.791438 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:08.791448 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:08.791460 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:31:08.791474 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:08.791537 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:31:08.791561 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:08.791801 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:08.791905 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 06:31:08.792141 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":392,"threads_started":1}
I20260809 06:31:08.796737 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:31:08.798844 9682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831: Bootstrap starting.
I20260809 06:31:08.802953 9682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:08.804205 9682 log.cc:824] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831: Log is configured to *not* fsync() on all Append() calls
I20260809 06:31:08.804591 9685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:08.801977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35905 (local address 127.8.131.62:35245)
0809 06:31:08.802171 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:08.802177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:08.802641 (+ 464us) server_negotiation.cc:407] Connection header received
0809 06:31:08.802776 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:08.802781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:08.802858 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:08.802992 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:08.803961 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:08.803968 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:08.803975 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 06:31:08.804003 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:08.804054 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 06:31:08.804075 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:08.804205 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:08.804261 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 06:31:08.804321 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 06:31:08.807199 9682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831: No bootstrap required, opened a new log
I20260809 06:31:08.814602 9682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7578a6939f084e19a3108742dd750831" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35245 } }
I20260809 06:31:08.814998 9682 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 06:31:08.815142 9682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7578a6939f084e19a3108742dd750831, State: Initialized, Role: FOLLOWER
I20260809 06:31:08.815819 9682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7578a6939f084e19a3108742dd750831" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35245 } }
I20260809 06:31:08.816068 9682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 06:31:08.816202 9682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 06:31:08.816408 9682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:08.818761 9682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7578a6939f084e19a3108742dd750831" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35245 } }
I20260809 06:31:08.819278 9682 leader_election.cc:302] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7578a6939f084e19a3108742dd750831; no voters:
I20260809 06:31:08.819978 9682 leader_election.cc:288] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 06:31:08.820147 9696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:31:08.820897 9696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [term 1 LEADER]: Becoming Leader. State: Replica: 7578a6939f084e19a3108742dd750831, State: Running, Role: LEADER
I20260809 06:31:08.821491 9696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7578a6939f084e19a3108742dd750831" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35245 } }
I20260809 06:31:08.822130 9682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:31:08.824604 9702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7578a6939f084e19a3108742dd750831. Latest consensus state: current_term: 1 leader_uuid: "7578a6939f084e19a3108742dd750831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7578a6939f084e19a3108742dd750831" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35245 } } }
I20260809 06:31:08.825260 9702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:08.824849 9700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7578a6939f084e19a3108742dd750831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7578a6939f084e19a3108742dd750831" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35245 } } }
I20260809 06:31:08.825855 9700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:08.828559 9712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:31:08.834261 9712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:31:08.844321 9712 catalog_manager.cc:1347] Generated new cluster ID: a83775fbedf94dcc97c863fceacebbcc
I20260809 06:31:08.844482 9712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:31:08.864367 9712 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:31:08.864817 9712 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:31:08.878554 9712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7578a6939f084e19a3108742dd750831: Generated new TSK 0
I20260809 06:31:08.879496 9712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:31:08.885147 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9586
W20260809 06:31:08.886394 9674 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45303
W20260809 06:31:08.887214 8716 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9586: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 06:31:08.887370 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9586
2026-08-09T06:31:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2155 ms)
[----------] 18 tests from AuthTokenIssuingTest (40104 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:31:09 dist-test-slave-mjjr krb5kdc[9740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:31:09 dist-test-slave-mjjr krb5kdc[9740](info): set up 2 sockets
Aug 09 06:31:09 dist-test-slave-mjjr krb5kdc[9740](info): commencing operation
krb5kdc: starting...
W20260809 06:31:10.216598 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.003s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:31:10 dist-test-slave-mjjr krb5kdc[9740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:31:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:31:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:31:10.522637 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:35333
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:37837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:35333,127.8.131.61:35895,127.8.131.60:35811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:31:10.860332 9754 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 06:31:10.860683 9754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:10.860793 9754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:10.860931 9754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:31:10.868350 9754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:10.868438 9754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:10.868536 9754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:10.868609 9754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:10.868731 9754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:10.878508 9754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:35333,127.8.131.61:35895,127.8.131.60:35811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:35333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:10.879875 9754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:10.881167 9754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:10.893149 9761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:10.893162 9760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:10.893620 9766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:10.893956 9754 server_base.cc:1034] running on GCE node
I20260809 06:31:10.894706 9754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:10.896116 9754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:10.897375 9754 hybrid_clock.cc:648] HybridClock initialized: now 1786257070897299 us; error 47 us; skew 500 ppm
I20260809 06:31:10.897692 9754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:10 dist-test-slave-mjjr krb5kdc[9740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257070, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:10.905845 9754 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:31:10.907300 9754 webserver.cc:460] Webserver started at http://127.8.131.62:46631/ using document root <none> and password file <none>
I20260809 06:31:10.908067 9754 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:10.908210 9754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:10.908542 9754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:10.911500 9754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "a9bf2b66672540a4a02e4f775d598401"
format_stamp: "Formatted at 2026-08-09 06:31:10 on dist-test-slave-mjjr"
server_key: "31172891e814baee63f34457b0e53184"
server_key_iv: "3168b40e944f1a952b64d76ad2959994"
server_key_version: "encryptionkey@0"
I20260809 06:31:10.912371 9754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "a9bf2b66672540a4a02e4f775d598401"
format_stamp: "Formatted at 2026-08-09 06:31:10 on dist-test-slave-mjjr"
server_key: "31172891e814baee63f34457b0e53184"
server_key_iv: "3168b40e944f1a952b64d76ad2959994"
server_key_version: "encryptionkey@0"
I20260809 06:31:10.917827 9754 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 06:31:10.921576 9773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:10.922780 9754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:31:10.922955 9754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "a9bf2b66672540a4a02e4f775d598401"
format_stamp: "Formatted at 2026-08-09 06:31:10 on dist-test-slave-mjjr"
server_key: "31172891e814baee63f34457b0e53184"
server_key_iv: "3168b40e944f1a952b64d76ad2959994"
server_key_version: "encryptionkey@0"
I20260809 06:31:10.923177 9754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:10.945150 9754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:10.949136 9754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:10.949427 9754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:10.968402 9754 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:35333
I20260809 06:31:10.968443 9862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:35333 every 8 connection(s)
I20260809 06:31:10.969755 9754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:31:10.971697 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9754
I20260809 06:31:10.971966 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:31:10.972913 8716 external_mini_cluster.cc:1428] Setting key 1b3d02bbc23e90c449d96e7d9acf1bae
I20260809 06:31:10.974506 9864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:31:10.985179 9864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:10 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257070, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.131.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.61@KRBTEST.COM" created.
I20260809 06:31:11.011188 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:10.989409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:35333 (local address 127.0.0.1:43220)
0809 06:31:10.990786 (+ 1377us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:10.990832 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:10.992700 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:10.993698 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:10.993703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:10.994318 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:10.994656 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:10.994673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:10.997123 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:10.997127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:10.998273 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:10.998292 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:10.999623 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:10.999629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:10.999980 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:11.000803 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:11.000836 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:11.003541 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:11.006973 (+ 3432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:11.006984 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:11.006994 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:11.007417 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:11.007964 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:11.007967 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:11.007970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:11.008134 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:11.008943 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:11.008949 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:11.009138 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 06:31:11.009512 (+ 374us) client_negotiation.cc:240] Negotiation successful
0809 06:31:11.010196 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"mutex_wait_us":272,"spinlock_wait_cycles":15616,"thread_start_us":819,"threads_started":1}
I20260809 06:31:11.011302 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:10.989409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43220 (local address 127.8.131.62:35333)
0809 06:31:10.990316 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:10.990333 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:10.990977 (+ 644us) server_negotiation.cc:407] Connection header received
0809 06:31:10.993089 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:10.993100 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:10.993438 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:10.993652 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:10.994930 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:10.996936 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:10.998436 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:10.999383 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:10.999579 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:11.003818 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:11.003826 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:11.003837 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:11.003878 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:11.006836 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:11.007595 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:11.007624 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:11.007630 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:11.007846 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:11.008235 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:11.008238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:11.008240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:11.008782 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:11.008975 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:11.009682 (+ 707us) server_negotiation.cc:299] Negotiation successful
0809 06:31:11.010038 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":652,"thread_start_us":409,"threads_started":1}
I20260809 06:31:11.014360 9864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:31:11.015340 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.014950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54544)
0809 06:31:11.015118 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.015226 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 06:31:11.015569 9778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
I20260809 06:31:11.017210 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.016837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54552)
0809 06:31:11.017030 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.017094 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 06:31:11.017417 9864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:35895: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab.
Entry for principal kudu/127.8.131.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab.
I20260809 06:31:11.050602 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.61:35895
--webserver_interface=127.8.131.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab
--principal=kudu/127.8.131.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:37837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:35333,127.8.131.61:35895,127.8.131.60:35811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:31:11.066742 9864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } attempt: 1
I20260809 06:31:11.068136 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.067508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54564)
0809 06:31:11.067779 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.067940 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 06:31:11.069322 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.068807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54580)
0809 06:31:11.069047 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.069152 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 06:31:11.069526 9864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:35895: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
I20260809 06:31:11.137807 9864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } attempt: 2
I20260809 06:31:11.139017 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.138540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54594)
0809 06:31:11.138758 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.138864 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 06:31:11.140228 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.139842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54604)
0809 06:31:11.140018 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.140090 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 06:31:11.140522 9864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:35895: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
I20260809 06:31:11.231772 9864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } attempt: 3
I20260809 06:31:11.232900 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.232479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54618)
0809 06:31:11.232673 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.232766 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 06:31:11.233847 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.233545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54630)
0809 06:31:11.233675 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.233736 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 06:31:11.234067 9864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:35895: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
I20260809 06:31:11.377496 9864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } attempt: 4
I20260809 06:31:11.378813 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.378275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54632)
0809 06:31:11.378597 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.378685 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260809 06:31:11.379901 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.379516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54636)
0809 06:31:11.379708 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.379786 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 06:31:11.380185 9864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:35895: Network error: Client connection negotiation failed: client connection to 127.8.131.61:35895: connect: Connection refused (error 111)
W20260809 06:31:11.389257 9872 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 06:31:11.389556 9872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:11.389621 9872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:11.389786 9872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:31:11.396932 9872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:11.397024 9872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:11.397081 9872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:11.397131 9872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:11.397241 9872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:11.406867 9872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:35333,127.8.131.61:35895,127.8.131.60:35811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab
--principal=kudu/127.8.131.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.61:35895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.131.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:11.408116 9872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:11.409373 9872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:11.420893 9882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:11.421005 9879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:11.420969 9880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:11.421972 9872 server_base.cc:1034] running on GCE node
I20260809 06:31:11.422762 9872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:11.424125 9872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:11.425370 9872 hybrid_clock.cc:648] HybridClock initialized: now 1786257071425297 us; error 43 us; skew 500 ppm
I20260809 06:31:11.425683 9872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:11 dist-test-slave-mjjr krb5kdc[9740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:11.429015 9872 init.cc:377] Logged in from keytab as kudu/127.8.131.61@KRBTEST.COM (short username kudu)
I20260809 06:31:11.430428 9872 webserver.cc:460] Webserver started at http://127.8.131.61:32965/ using document root <none> and password file <none>
I20260809 06:31:11.431037 9872 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:11.431125 9872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:11.431396 9872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:11.434267 9872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/data/instance:
uuid: "2c1f62926c5c4370a27d605ebd618139"
format_stamp: "Formatted at 2026-08-09 06:31:11 on dist-test-slave-mjjr"
server_key: "4c8d2c12c02d9ebe38d2be5538c1d7f5"
server_key_iv: "d60def5e6c1226fa37e2b7931479a967"
server_key_version: "encryptionkey@0"
I20260809 06:31:11.434993 9872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/wal/instance:
uuid: "2c1f62926c5c4370a27d605ebd618139"
format_stamp: "Formatted at 2026-08-09 06:31:11 on dist-test-slave-mjjr"
server_key: "4c8d2c12c02d9ebe38d2be5538c1d7f5"
server_key_iv: "d60def5e6c1226fa37e2b7931479a967"
server_key_version: "encryptionkey@0"
I20260809 06:31:11.440006 9872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 06:31:11.443202 9889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:11.444700 9872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:31:11.444839 9872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/wal
uuid: "2c1f62926c5c4370a27d605ebd618139"
format_stamp: "Formatted at 2026-08-09 06:31:11 on dist-test-slave-mjjr"
server_key: "4c8d2c12c02d9ebe38d2be5538c1d7f5"
server_key_iv: "d60def5e6c1226fa37e2b7931479a967"
server_key_version: "encryptionkey@0"
I20260809 06:31:11.445019 9872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:11.457775 9872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:11.461601 9872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:11.461838 9872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:11.480448 9872 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.61:35895
I20260809 06:31:11.480482 9979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.61:35895 every 8 connection(s)
I20260809 06:31:11.481837 9872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
I20260809 06:31:11.483310 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9872
I20260809 06:31:11.483527 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-1/wal/instance
I20260809 06:31:11.484169 8716 external_mini_cluster.cc:1428] Setting key 66a70638ea07b49412f8947f12ebfddf
I20260809 06:31:11.486444 9982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:31:11.496389 9982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:11 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.131.60@KRBTEST.COM; defaulting to no policy
I20260809 06:31:11.519200 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.499675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.8.131.62:35333)
0809 06:31:11.499910 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:11.499917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:11.501248 (+ 1331us) server_negotiation.cc:407] Connection header received
0809 06:31:11.503166 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:11.503175 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:11.503275 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:11.503419 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:11.504886 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.506089 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.507564 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.508422 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.508565 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:11.512303 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:11.512312 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:11.512319 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:11.512373 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:11.516175 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:11.516861 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:11.516867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:11.516870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:11.517028 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:11.517474 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:11.517482 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:11.517486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:11.517837 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:11.518003 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:11.518717 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 06:31:11.518890 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 06:31:11.520241 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.500196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:35333 (local address 127.0.0.1:43232)
0809 06:31:11.501068 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.501112 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:11.502871 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:11.503584 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:11.503591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:11.504173 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:11.504707 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:11.504723 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.506244 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.506248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:11.507417 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:11.507430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.508614 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.508619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:11.508815 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:11.509558 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:11.509580 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:11.512071 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:11.516308 (+ 4237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:11.516315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:11.516326 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:11.516726 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:11.517171 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:11.517175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:11.517177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:11.517342 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:11.518022 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:11.518031 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:11.518362 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 06:31:11.518589 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 06:31:11.518942 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":393,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.60@KRBTEST.COM" created.
I20260809 06:31:11.522006 9982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:11 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:31:11.537876 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.523010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.8.131.61:35895)
0809 06:31:11.523634 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:11.523685 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:11.523722 (+ 37us) server_negotiation.cc:407] Connection header received
0809 06:31:11.523767 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:11.523772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:11.523935 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:11.524089 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:11.524820 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.525855 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.527087 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.527608 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.527778 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:11.531705 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:11.531722 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:11.531741 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:11.531785 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:11.534998 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:11.535799 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:11.535805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:11.535807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:11.535924 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:11.536339 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:11.536342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:11.536344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:11.536656 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:11.536800 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:11.537282 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 06:31:11.537563 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":488,"thread_start_us":357,"threads_started":1}
I20260809 06:31:11.537981 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.522638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54648)
0809 06:31:11.522831 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.522849 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:11.522957 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:11.524079 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:11.524083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:11.524521 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:11.524645 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:11.524653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.525998 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.526003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:11.526960 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:11.526971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.527783 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.527788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:11.528000 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:11.528613 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:11.528640 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:11.531432 (+ 2792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:11.535126 (+ 3694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:11.535134 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:11.535141 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:11.535682 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:11.536043 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:11.536051 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:11.536058 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:11.536238 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:11.536793 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:11.536800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:11.536979 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 06:31:11.537137 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 06:31:11.537473 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 06:31:11.540336 9982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:31:11.541399 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.540985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45548)
0809 06:31:11.541149 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.541255 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 06:31:11.541632 9895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
I20260809 06:31:11.543159 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.542840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45556)
0809 06:31:11.542993 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.543065 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 06:31:11.543393 9982 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:35811: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab.
Entry for principal kudu/127.8.131.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab.
I20260809 06:31:11.558696 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.60:35811
--webserver_interface=127.8.131.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab
--principal=kudu/127.8.131.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:37837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:35333,127.8.131.61:35895,127.8.131.60:35811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:31:11.592669 9982 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } attempt: 1
I20260809 06:31:11.593933 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.593459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45566)
0809 06:31:11.593662 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.593770 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 06:31:11.594992 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.594626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45576)
0809 06:31:11.594791 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.594885 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 06:31:11.595211 9982 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:35811: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
I20260809 06:31:11.663461 9982 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } attempt: 2
I20260809 06:31:11.664737 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.664250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45586)
0809 06:31:11.664478 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.664581 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 06:31:11.665781 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.665442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45588)
0809 06:31:11.665602 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.665667 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 06:31:11.665975 9982 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:35811: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
I20260809 06:31:11.679509 9864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } attempt: 5
Aug 09 06:31:11 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257070, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:31:11.694203 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.680359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54654)
0809 06:31:11.680544 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.680566 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:11.680686 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:11.681041 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:11.681044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:11.681454 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:11.681632 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:11.681641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.683040 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.683044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:11.683947 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:11.683958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.684882 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.684887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:11.685016 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:11.685655 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:11.685673 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:11.687907 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:11.691903 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:11.691907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:11.691909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:11.692283 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:11.692738 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:11.692741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:11.692743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:11.692870 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:11.693527 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:11.693530 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:11.693649 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 06:31:11.693767 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 06:31:11.693957 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 06:31:11.694474 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.680476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54654 (local address 127.8.131.61:35895)
0809 06:31:11.680669 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:11.680675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:11.680700 (+ 25us) server_negotiation.cc:407] Connection header received
0809 06:31:11.680792 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:11.680797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:11.680876 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:11.681000 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:11.681792 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.682895 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.684097 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:11.684733 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:11.684864 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:11.688099 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:11.688106 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:11.688110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:11.688148 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:11.691766 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:11.692415 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:11.692423 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:11.692428 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:11.692610 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:11.692997 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:11.693005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:11.693010 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:11.693396 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:11.693565 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:11.693924 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 06:31:11.694167 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 06:31:11.695562 9864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:31:11.696591 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.696203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45592)
0809 06:31:11.696375 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.696456 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 06:31:11.697527 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.697236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45604)
0809 06:31:11.697361 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.697441 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 06:31:11.697790 9864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:35811: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
I20260809 06:31:11.749086 9864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } attempt: 1
I20260809 06:31:11.750414 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.749900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45616)
0809 06:31:11.750129 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.750231 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 06:31:11.751475 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.751135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45632)
0809 06:31:11.751289 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.751369 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 06:31:11.751760 9864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:35811: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
I20260809 06:31:11.757261 9982 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } attempt: 3
I20260809 06:31:11.758229 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.757795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45642)
0809 06:31:11.758006 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.758091 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 06:31:11.759281 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.758903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45644)
0809 06:31:11.759117 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.759184 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260809 06:31:11.759501 9982 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:35811: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
I20260809 06:31:11.820036 9864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } attempt: 2
I20260809 06:31:11.821318 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.820814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45648)
0809 06:31:11.821056 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.821153 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 06:31:11.822393 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.822064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45656)
0809 06:31:11.822204 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.822277 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 06:31:11.822711 9864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:35811: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
I20260809 06:31:11.902729 9982 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } attempt: 4
I20260809 06:31:11.904137 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.903487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45666)
0809 06:31:11.903774 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.903888 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 06:31:11.905225 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.904834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45680)
0809 06:31:11.905010 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.905096 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 06:31:11.905457 9982 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:35811: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
W20260809 06:31:11.910972 9990 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 06:31:11.911279 9990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:11.911350 9990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:11.911432 9990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:31:11.918509 9990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:11.918591 9990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:11.918648 9990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:11.918696 9990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:11.918761 9990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:11.928875 9990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:37837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:35333,127.8.131.61:35895,127.8.131.60:35811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab
--principal=kudu/127.8.131.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.60:35811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.131.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:11.928965 9864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } attempt: 3
I20260809 06:31:11.930100 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.929646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45692)
0809 06:31:11.929869 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.929955 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 06:31:11.930425 9990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:11.931169 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:11.930801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45704)
0809 06:31:11.930962 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:11.931047 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 06:31:11.931391 9864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:35811: Network error: Client connection negotiation failed: client connection to 127.8.131.60:35811: connect: Connection refused (error 111)
I20260809 06:31:11.932027 9990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:11.943961 9998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:11.943961 9999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:11.943984 10006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:11.945952 9990 server_base.cc:1034] running on GCE node
I20260809 06:31:11.946702 9990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:11.948206 9990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:11.949463 9990 hybrid_clock.cc:648] HybridClock initialized: now 1786257071949388 us; error 44 us; skew 500 ppm
I20260809 06:31:11.949815 9990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:11 dist-test-slave-mjjr krb5kdc[9740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:11.953042 9990 init.cc:377] Logged in from keytab as kudu/127.8.131.60@KRBTEST.COM (short username kudu)
I20260809 06:31:11.954438 9990 webserver.cc:460] Webserver started at http://127.8.131.60:39935/ using document root <none> and password file <none>
I20260809 06:31:11.955127 9990 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:11.955257 9990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:11.955583 9990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:11.958515 9990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/data/instance:
uuid: "1637ac1d18cc4a5894ef8453b3245b88"
format_stamp: "Formatted at 2026-08-09 06:31:11 on dist-test-slave-mjjr"
server_key: "c9520fe1d96acdc9bb177fb10013ad2f"
server_key_iv: "ec1232e0dbf0ffb7fa8d304e0f3a1e7a"
server_key_version: "encryptionkey@0"
I20260809 06:31:11.959311 9990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/wal/instance:
uuid: "1637ac1d18cc4a5894ef8453b3245b88"
format_stamp: "Formatted at 2026-08-09 06:31:11 on dist-test-slave-mjjr"
server_key: "c9520fe1d96acdc9bb177fb10013ad2f"
server_key_iv: "ec1232e0dbf0ffb7fa8d304e0f3a1e7a"
server_key_version: "encryptionkey@0"
I20260809 06:31:11.964447 9990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 06:31:11.968020 10016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:11.969544 9990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 06:31:11.969709 9990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/wal
uuid: "1637ac1d18cc4a5894ef8453b3245b88"
format_stamp: "Formatted at 2026-08-09 06:31:11 on dist-test-slave-mjjr"
server_key: "c9520fe1d96acdc9bb177fb10013ad2f"
server_key_iv: "ec1232e0dbf0ffb7fa8d304e0f3a1e7a"
server_key_version: "encryptionkey@0"
I20260809 06:31:11.969934 9990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:11.997829 9990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:12.001780 9990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:12.002079 9990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:12.020411 9990 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.60:35811
I20260809 06:31:12.020429 10089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.60:35811 every 8 connection(s)
I20260809 06:31:12.021785 9990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
I20260809 06:31:12.025439 10090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:31:12.030591 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 9990
I20260809 06:31:12.030787 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786256699676692-8716-0/minicluster-data/master-2/wal/instance
I20260809 06:31:12.031293 8716 external_mini_cluster.cc:1428] Setting key e37825cbf340e7e3913d559b2a398705
I20260809 06:31:12.032651 10090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:12 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
Aug 09 06:31:12 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
I20260809 06:31:12.054055 10095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.035723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.8.131.62:35333)
0809 06:31:12.036815 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.036821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.037076 (+ 255us) server_negotiation.cc:407] Connection header received
0809 06:31:12.037694 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.037698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.037758 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.037844 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.039506 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.040710 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.042837 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.043448 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.043568 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.047699 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.047707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.047718 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.047751 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.050850 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.051594 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.051600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.051604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.051809 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.052216 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.052221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.052225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.052512 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.052661 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.053182 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.053415 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":821,"threads_started":1}
I20260809 06:31:12.054757 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.035693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:35333 (local address 127.0.0.1:43238)
0809 06:31:12.036925 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:12.036967 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:12.037153 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:12.037957 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:12.037966 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:12.038806 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:12.039338 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.039354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.040883 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.040887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.042711 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.042723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.043704 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.043713 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.043886 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.044638 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:12.044669 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:12.047458 (+ 2789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:12.051021 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.051033 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.051044 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.051462 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.051922 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.051928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.051932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.052095 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.052650 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:12.052658 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:12.052849 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 06:31:12.053095 (+ 246us) client_negotiation.cc:240] Negotiation successful
0809 06:31:12.053603 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":2,"thread_start_us":398,"threads_started":1}
I20260809 06:31:12.054776 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.033271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.8.131.60:35811)
0809 06:31:12.034043 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.034057 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.034105 (+ 48us) server_negotiation.cc:407] Connection header received
0809 06:31:12.036694 (+ 2589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.036715 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.037046 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.037292 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.039036 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.040894 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.042670 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.043430 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.043602 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.048178 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.048186 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.048215 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.048243 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.051081 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.051765 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.051778 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.051780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.051931 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.052312 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.052318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.052321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.052785 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.053100 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.053419 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.053731 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":541,"thread_start_us":337,"threads_started":1}
I20260809 06:31:12.056348 10090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:12 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
Aug 09 06:31:12 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:31:12.070962 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.057001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:35895 (local address 127.0.0.1:54662)
0809 06:31:12.057177 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:12.057195 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:12.057292 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:12.057942 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:12.057947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:12.058433 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:12.058657 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.058670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.059750 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.059756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.060903 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.060918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.061621 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.061627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.061858 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.062464 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:12.062489 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:12.064912 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:12.068435 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.068440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.068445 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.068976 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.069356 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.069361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.069364 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.069539 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.070026 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:12.070030 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:12.070178 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 06:31:12.070317 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 06:31:12.070585 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 06:31:12.071002 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.057248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54662 (local address 127.8.131.61:35895)
0809 06:31:12.057453 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.057460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.057479 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:31:12.057545 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.057551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.057650 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.057750 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.058818 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.059567 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.061062 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.061470 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.061565 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.065104 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.065109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.065112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.065142 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.068299 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.069093 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.069096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.069099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.069231 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.069652 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.069656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.069658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.069908 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.070059 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.070432 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.070631 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 06:31:12.072265 10090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:31:12.072651 10095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.057750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.8.131.62:35333)
0809 06:31:12.057918 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.057922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.057937 (+ 15us) server_negotiation.cc:407] Connection header received
0809 06:31:12.057974 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.057977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.058037 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.058110 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.059476 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.060232 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.061744 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.062238 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.062375 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.066833 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.066838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.066841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.066871 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.069889 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.070523 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.070535 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.070537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.070648 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.071019 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.071025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.071029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.071427 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.071740 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.072100 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.072335 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 06:31:12 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
Aug 09 06:31:12 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:31:12.086061 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.073019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45728)
0809 06:31:12.073198 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:12.073217 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:12.073322 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:12.073586 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:12.073591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:12.074026 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:12.074246 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.074262 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.075344 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.075348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.076254 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.076266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.077163 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.077169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.077401 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.077908 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:12.077929 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:12.079915 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:12.083533 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.083539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.083542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.084078 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.084423 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.084430 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.084436 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.084614 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.085093 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:12.085097 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:12.085237 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 06:31:12.085366 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 06:31:12.085668 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 06:31:12.086124 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.073212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45728 (local address 127.8.131.60:35811)
0809 06:31:12.073340 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.073344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.073358 (+ 14us) server_negotiation.cc:407] Connection header received
0809 06:31:12.073405 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.073408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.073456 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.073527 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.074401 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.075221 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.076403 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.077026 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.077116 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.080088 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.080094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.080099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.080135 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.083413 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.084175 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.084179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.084181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.084304 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.084724 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.084728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.084729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.084973 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.085113 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.085479 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.085819 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":29}
I20260809 06:31:12.090822 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.074527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.8.131.61:35895)
0809 06:31:12.075415 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.075419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.075431 (+ 12us) server_negotiation.cc:407] Connection header received
0809 06:31:12.075466 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.075469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.075515 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.075616 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.076930 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.077994 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.079154 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.079617 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.079798 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.085095 (+ 5297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.085100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.085103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.085131 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.088328 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.088938 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.088949 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.088952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.089078 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.089463 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.089468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.089471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.089718 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.089993 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.090253 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.090496 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787}
I20260809 06:31:12.091585 10090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88: Bootstrap starting.
I20260809 06:31:12.095050 10090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:12.095606 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:31:12.096230 10090 log.cc:824] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88: Log is configured to *not* fsync() on all Append() calls
I20260809 06:31:12.099154 10090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88: No bootstrap required, opened a new log
I20260809 06:31:12.098290 9821 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a9bf2b66672540a4a02e4f775d598401"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43250
I20260809 06:31:12.108595 9864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } attempt: 4
I20260809 06:31:12.109915 10090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } }
I20260809 06:31:12.110306 10090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1637ac1d18cc4a5894ef8453b3245b88, State: Initialized, Role: FOLLOWER
I20260809 06:31:12.110793 10090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } }
I20260809 06:31:12.112072 10099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.112484 10099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:12.113112 10090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 06:31:12 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257070, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
I20260809 06:31:12.124904 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.109211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45740)
0809 06:31:12.109394 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:12.109414 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:12.109524 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:12.109851 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:12.109856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:12.110323 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:12.110568 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.110580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.111775 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.111781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.113275 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.113292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.114115 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.114121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.114328 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.115148 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:12.115175 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:12.117868 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:12.122006 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.122011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.122015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.122412 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.122832 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.122840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.122846 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.123024 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.123887 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:12.123893 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:12.124061 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 06:31:12.124198 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 06:31:12.124500 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 06:31:12.125583 10114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:31:12.125700 10114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 06:31:12.126150 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.109291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45740 (local address 127.8.131.60:35811)
0809 06:31:12.109490 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.109496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.109514 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:31:12.109653 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.109658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.109722 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.109819 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.110715 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.111587 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.113418 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.113971 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.114103 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.118044 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.118049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.118052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.118109 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.121888 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.122536 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.122543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.122547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.122704 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.123184 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.123189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.123192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.123697 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.125308 (+ 1611us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.125438 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.125679 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 06:31:12.130602 9864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401: Bootstrap starting.
I20260809 06:31:12.122890 9990 master_runner.cc:186] Error getting master registration for 127.8.131.62:35333: OK, instance_id { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" instance_seqno: 1786257070962781 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 06:31:12.133349 9990 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 06:31:12.134265 9864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:12.135490 9864 log.cc:824] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401: Log is configured to *not* fsync() on all Append() calls
I20260809 06:31:12.138726 9864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401: No bootstrap required, opened a new log
I20260809 06:31:12.143887 9864 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } }
I20260809 06:31:12.144229 9864 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bf2b66672540a4a02e4f775d598401, State: Initialized, Role: FOLLOWER
I20260809 06:31:12.144649 9864 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } }
I20260809 06:31:12.145666 10118 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.146095 10118 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:12.146765 9864 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:31:12.146961 9821 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 06:31:12.147161 9821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:12.150724 9821 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } }
I20260809 06:31:12.153430 9821 leader_election.cc:288] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [CANDIDATE]: Term 1 election: Requested vote from peers 2c1f62926c5c4370a27d605ebd618139 (127.8.131.61:35895), 1637ac1d18cc4a5894ef8453b3245b88 (127.8.131.60:35811)
I20260809 06:31:12.153925 10056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf2b66672540a4a02e4f775d598401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1637ac1d18cc4a5894ef8453b3245b88"
I20260809 06:31:12.154523 10056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:12.158998 10056 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bf2b66672540a4a02e4f775d598401 in term 1.
I20260809 06:31:12.159894 9778 leader_election.cc:302] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1637ac1d18cc4a5894ef8453b3245b88, a9bf2b66672540a4a02e4f775d598401; no voters:
I20260809 06:31:12.160475 10118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:31:12.161099 10118 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [term 1 LEADER]: Becoming Leader. State: Replica: a9bf2b66672540a4a02e4f775d598401, State: Running, Role: LEADER
I20260809 06:31:12.161721 10118 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } }
I20260809 06:31:12.173741 10131 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf2b66672540a4a02e4f775d598401. Latest consensus state: current_term: 1 leader_uuid: "a9bf2b66672540a4a02e4f775d598401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.174173 10131 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:12.189186 10136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:31:12.199383 10136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 06:31:12.202437 9785 tablet.cc:2369] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:31:12.204658 9982 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } attempt: 5
I20260809 06:31:12.220628 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.187967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.8.131.61:35895)
0809 06:31:12.188188 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.188198 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.200195 (+ 11997us) server_negotiation.cc:407] Connection header received
0809 06:31:12.201125 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.201134 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.201238 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.201327 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.203581 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.204651 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.206718 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.207628 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.207783 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.211688 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.211696 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.211702 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.211743 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.215501 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.217424 (+ 1923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.217430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.217434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.217605 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.218393 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.218399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.218402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.218796 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.219024 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.219999 (+ 975us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.220247 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 06:31:12.235092 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.200454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.8.131.60:35811)
0809 06:31:12.200675 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.200684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.210341 (+ 9657us) server_negotiation.cc:407] Connection header received
0809 06:31:12.210751 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.210757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.210860 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.210937 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.214277 (+ 3340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.215400 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.218951 (+ 3551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.219756 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.219871 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.225283 (+ 5412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.225290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.225294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.225338 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.231568 (+ 6230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.232511 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.232517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.232521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.232674 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.233140 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.233147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.233151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.233494 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.233723 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.234313 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.234714 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 06:31:12.237106 10095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.186092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43258 (local address 127.8.131.62:35333)
0809 06:31:12.186347 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.186353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.187388 (+ 1035us) server_negotiation.cc:407] Connection header received
0809 06:31:12.187706 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.187712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.187795 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.187871 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.190251 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.191511 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.193552 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.194279 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.194378 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.198119 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.198126 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.198131 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.198171 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.201901 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.203018 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.203026 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.203029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.203188 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.205612 (+ 2424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.205620 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.205625 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.208694 (+ 3069us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.223836 (+ 15142us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.236446 (+ 12610us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.236714 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:31:12.237746 10056 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [term 1 FOLLOWER]: Refusing update from remote peer a9bf2b66672540a4a02e4f775d598401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
Aug 09 06:31:12 dist-test-slave-mjjr krb5kdc[9740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
I20260809 06:31:12.238919 10131 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:31:12.224833 9754 master_runner.cc:186] Error getting master registration for 127.8.131.61:35895: OK, instance_id { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" instance_seqno: 1786257071475164 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 06:31:12.239408 9754 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 06:31:12.249032 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.205205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:35811 (local address 127.0.0.1:45760)
0809 06:31:12.205572 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:12.205592 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:12.205685 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:12.229618 (+ 23933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:12.229623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:12.230115 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:12.230310 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.230323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.232188 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.232193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.233606 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:12.233621 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.234589 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.234595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:12.234804 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.235859 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:12.235892 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:12.239995 (+ 4103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:12.245924 (+ 5929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.245930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.245933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.246496 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.247068 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:12.247073 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:12.247077 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:12.247257 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:12.247995 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:12.248000 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:12.248161 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 06:31:12.248315 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 06:31:12.248616 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219}
I20260809 06:31:12.250190 10150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:12.205872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.8.131.60:35811)
0809 06:31:12.229169 (+ 23297us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.229175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.229201 (+ 26us) server_negotiation.cc:407] Connection header received
0809 06:31:12.229259 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.229264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.229334 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.230478 (+ 1144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.230722 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.232020 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.233792 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:12.234412 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:12.235096 (+ 684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:12.240332 (+ 5236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:12.240340 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:12.240346 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:12.240416 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:12.245763 (+ 5347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.246671 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.246675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.246679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.246932 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:12.247410 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:12.247414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:12.247418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:12.247832 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:12.249272 (+ 1440us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:12.249401 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 06:31:12.249647 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23121,"thread_start_us":23525,"threads_started":1}
W20260809 06:31:12.254223 10022 tablet.cc:2369] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:31:12.255970 9982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139: Bootstrap starting.
I20260809 06:31:12.260131 9982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:12.261401 9982 log.cc:824] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139: Log is configured to *not* fsync() on all Append() calls
I20260809 06:31:12.274760 9982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139: No bootstrap required, opened a new log
I20260809 06:31:12.287684 9982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } }
I20260809 06:31:12.288122 9982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c1f62926c5c4370a27d605ebd618139, State: Initialized, Role: FOLLOWER
I20260809 06:31:12.288666 9982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } }
I20260809 06:31:12.291144 10099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf2b66672540a4a02e4f775d598401. Latest consensus state: current_term: 1 leader_uuid: "a9bf2b66672540a4a02e4f775d598401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.291553 10099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:12.296578 10154 mvcc.cc:205] Tried to move back new op lower bound from 7316508967853498368 to 7316508967579852800. Current Snapshot: MvccSnapshot[applied={T|T < 7316508967853498368}]
I20260809 06:31:12.299221 10118 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9bf2b66672540a4a02e4f775d598401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.299553 10118 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:12.301367 10161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.304790 10136 catalog_manager.cc:1347] Generated new cluster ID: 2184f86fecfa4b0aad16b2a47de47d4f
I20260809 06:31:12.304893 10136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:31:12.314952 10161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:12.306320 9934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:12.318962 9934 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [term 1 FOLLOWER]: Refusing update from remote peer a9bf2b66672540a4a02e4f775d598401: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:31:12.306507 9936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf2b66672540a4a02e4f775d598401" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2c1f62926c5c4370a27d605ebd618139"
I20260809 06:31:12.319999 10131 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:31:12.322829 9936 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bf2b66672540a4a02e4f775d598401 in term 1.
I20260809 06:31:12.304069 9982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:31:12.342016 10099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9bf2b66672540a4a02e4f775d598401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.342303 10099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:12.354233 10136 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:31:12.356254 10136 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:31:12.373649 10161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf2b66672540a4a02e4f775d598401. Latest consensus state: current_term: 1 leader_uuid: "a9bf2b66672540a4a02e4f775d598401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.373993 10161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:12.374404 10167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9bf2b66672540a4a02e4f775d598401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.374897 10167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:12.384774 10118 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9bf2b66672540a4a02e4f775d598401" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9bf2b66672540a4a02e4f775d598401" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 35333 } } peers { permanent_uuid: "2c1f62926c5c4370a27d605ebd618139" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 35895 } } peers { permanent_uuid: "1637ac1d18cc4a5894ef8453b3245b88" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 35811 } } }
I20260809 06:31:12.385072 10118 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:12.396862 10136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9bf2b66672540a4a02e4f775d598401: Generated new TSK 0
I20260809 06:31:12.398072 10136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:31:12.400344 10193 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 06:31:12.482121 10095 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.479296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.8.131.62:35333)
0809 06:31:12.479536 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.479543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.480144 (+ 601us) server_negotiation.cc:407] Connection header received
0809 06:31:12.480457 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.480463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.480547 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.480626 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.481863 (+ 1237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43274: BlockingRecv error: recv got EOF from 127.0.0.1:43274 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 06:31:12.484253 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.480599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.8.131.61:35895)
0809 06:31:12.480793 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.480803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.482705 (+ 1902us) server_negotiation.cc:407] Connection header received
0809 06:31:12.482964 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.482970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.483041 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.483104 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.484046 (+ 942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54686: BlockingRecv error: recv got EOF from 127.0.0.1:54686 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 06:31:12.492573 10150 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.490390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45772 (local address 127.8.131.60:35811)
0809 06:31:12.490542 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.490548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.491056 (+ 508us) server_negotiation.cc:407] Connection header received
0809 06:31:12.491208 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.491214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.491294 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.491407 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.492341 (+ 934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45772: BlockingRecv error: recv got EOF from 127.0.0.1:45772 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 06:31:12.514200 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.511973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.8.131.61:35895)
0809 06:31:12.512141 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.512147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.512665 (+ 518us) server_negotiation.cc:407] Connection header received
0809 06:31:12.512830 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.512836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.512959 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.513070 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.513925 (+ 855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54690: BlockingRecv error: recv got EOF from 127.0.0.1:54690 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 06:31:12.527715 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.525612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.8.131.61:35895)
0809 06:31:12.525770 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.525777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.526175 (+ 398us) server_negotiation.cc:407] Connection header received
0809 06:31:12.526345 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.526350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.526436 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.526551 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.527457 (+ 906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54698: BlockingRecv error: recv got EOF from 127.0.0.1:54698 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 06:31:12.534958 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.533137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54706 (local address 127.8.131.61:35895)
0809 06:31:12.533281 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.533287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.533641 (+ 354us) server_negotiation.cc:407] Connection header received
0809 06:31:12.533813 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.533817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.533890 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.533986 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.534747 (+ 761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54706: BlockingRecv error: recv got EOF from 127.0.0.1:54706 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 06:31:12.547438 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.545288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.8.131.61:35895)
0809 06:31:12.545483 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.545489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.546032 (+ 543us) server_negotiation.cc:407] Connection header received
0809 06:31:12.546193 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.546198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.546281 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.546387 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.547241 (+ 854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54718: BlockingRecv error: recv got EOF from 127.0.0.1:54718 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260809 06:31:12.560809 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.558677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.8.131.61:35895)
0809 06:31:12.558845 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.558851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.559355 (+ 504us) server_negotiation.cc:407] Connection header received
0809 06:31:12.559548 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.559553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.559664 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.559770 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.560584 (+ 814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54720: BlockingRecv error: recv got EOF from 127.0.0.1:54720 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 06:31:12.582932 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.580509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.8.131.61:35895)
0809 06:31:12.580701 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.580707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.581281 (+ 574us) server_negotiation.cc:407] Connection header received
0809 06:31:12.581500 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.581505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.581620 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.581741 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.582672 (+ 931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54728: BlockingRecv error: recv got EOF from 127.0.0.1:54728 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 06:31:12.624053 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.621398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54738 (local address 127.8.131.61:35895)
0809 06:31:12.621623 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.621630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.622224 (+ 594us) server_negotiation.cc:407] Connection header received
0809 06:31:12.622487 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.622495 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.622602 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.622729 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.623767 (+ 1038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54738: BlockingRecv error: recv got EOF from 127.0.0.1:54738 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 06:31:12.699689 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.697339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.8.131.61:35895)
0809 06:31:12.697542 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.697549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.698074 (+ 525us) server_negotiation.cc:407] Connection header received
0809 06:31:12.698279 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.698286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.698393 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.698523 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.699415 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54748: BlockingRecv error: recv got EOF from 127.0.0.1:54748 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 06:31:12.833009 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:12.831195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54758 (local address 127.8.131.61:35895)
0809 06:31:12.831390 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:12.831395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:12.831713 (+ 318us) server_negotiation.cc:407] Connection header received
0809 06:31:12.831938 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:12.831942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:12.832028 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:12.832132 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:12.832807 (+ 675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54758: BlockingRecv error: recv got EOF from 127.0.0.1:54758 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 06:31:13.094657 9986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:13.092752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54768 (local address 127.8.131.61:35895)
0809 06:31:13.092982 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:13.092990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:13.093196 (+ 206us) server_negotiation.cc:407] Connection header received
0809 06:31:13.093462 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:13.093466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:13.093537 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:13.093648 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:13.094434 (+ 786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54768: BlockingRecv error: recv got EOF from 127.0.0.1:54768 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 06:31:13.127903 10114 catalog_manager.cc:1259] Loaded cluster ID: 2184f86fecfa4b0aad16b2a47de47d4f
I20260809 06:31:13.128033 10114 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88: loading cluster ID for follower catalog manager: success
I20260809 06:31:13.130568 10114 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88: acquiring CA information for follower catalog manager: success
I20260809 06:31:13.132359 10114 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1637ac1d18cc4a5894ef8453b3245b88: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:31:13.404707 10193 catalog_manager.cc:1259] Loaded cluster ID: 2184f86fecfa4b0aad16b2a47de47d4f
I20260809 06:31:13.404909 10193 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139: loading cluster ID for follower catalog manager: success
I20260809 06:31:13.407516 10193 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139: acquiring CA information for follower catalog manager: success
I20260809 06:31:13.409343 10193 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2c1f62926c5c4370a27d605ebd618139: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:31:13.620898 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:13.613387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.8.131.61:35895)
0809 06:31:13.614390 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:13.614396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:13.614414 (+ 18us) server_negotiation.cc:407] Connection header received
0809 06:31:13.614488 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:13.614491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:13.614726 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:13.614862 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:13.615315 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.616270 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.618310 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.618777 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.618882 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:13.619533 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:13.620159 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:13.620273 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:13.620436 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 06:31:13.620511 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":662,"threads_started":1}
I20260809 06:31:13.634462 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:13.628159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45776 (local address 127.8.131.60:35811)
0809 06:31:13.628681 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:13.628686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:13.628702 (+ 16us) server_negotiation.cc:407] Connection header received
0809 06:31:13.628789 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:13.628792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:13.628939 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:13.629074 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:13.629589 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.630316 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.632247 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.632668 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.632757 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:13.633315 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:13.633795 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:13.633899 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:13.634062 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 06:31:13.634124 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":278,"threads_started":1}
I20260809 06:31:13.651962 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:13.642975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.8.131.61:35895)
0809 06:31:13.643150 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:13.643158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:13.643179 (+ 21us) server_negotiation.cc:407] Connection header received
0809 06:31:13.643401 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:13.643408 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:13.643495 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:13.643561 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:13.644242 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.645774 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.648799 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.649815 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.649942 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:13.650825 (+ 883us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:13.651015 (+ 190us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:13.651090 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:13.651367 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 06:31:13.651473 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:31:13.659180 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:13.643941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.8.131.60:35811)
0809 06:31:13.644103 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:13.644111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:13.652861 (+ 8750us) server_negotiation.cc:407] Connection header received
0809 06:31:13.653172 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:13.653181 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:13.653311 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:13.653378 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:13.654091 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.655323 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.657156 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.657792 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.657935 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:13.658412 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:13.658553 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:13.658655 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:13.658818 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 06:31:13.658901 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:31:13.662724 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:13.642015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.8.131.62:35333)
0809 06:31:13.656911 (+ 14896us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:13.656919 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:13.656939 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:31:13.657057 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:13.657064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:13.657271 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:13.657460 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:13.657911 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.658870 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.660794 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:13.661236 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:13.661364 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:13.661815 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:13.662130 (+ 315us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:13.662244 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:13.662382 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 06:31:13.662471 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14766,"thread_start_us":14670,"threads_started":1}
I20260809 06:31:13.668571 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9754
I20260809 06:31:13.669857 9857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:31:13.780656 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9754
I20260809 06:31:13.808972 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9872
I20260809 06:31:13.810108 9967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:31:13.920009 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9872
I20260809 06:31:13.947114 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 9990
I20260809 06:31:13.948283 10085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:31:14.055594 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 9990
2026-08-09T06:31:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5177 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 06:31:14 dist-test-slave-mjjr krb5kdc[10321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 06:31:14 dist-test-slave-mjjr krb5kdc[10321](info): set up 2 sockets
Aug 09 06:31:14 dist-test-slave-mjjr krb5kdc[10321](info): commencing operation
krb5kdc: starting...
W20260809 06:31:15.381525 8716 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/test-user.keytab.
Aug 09 06:31:15 dist-test-slave-mjjr krb5kdc[10321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T06:31:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T06:31:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.131.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
Entry for principal kudu/127.8.131.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab.
I20260809 06:31:15.684376 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:40167
--webserver_interface=127.8.131.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:45871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:40167,127.8.131.61:43071,127.8.131.60:44091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 06:31:16.075083 10341 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 06:31:16.075484 10341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:16.075614 10341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:16.075811 10341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:31:16.083307 10341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:16.083398 10341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:16.083546 10341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:16.083632 10341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:16.083799 10341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:16.093755 10341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:45871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:40167,127.8.131.61:43071,127.8.131.60:44091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.62.keytab
--principal=kudu/127.8.131.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.62:40167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.131.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:16.095054 10341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:16.096437 10341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:16.108822 10355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:16.108889 10348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:16.108920 10350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:16.109894 10341 server_base.cc:1034] running on GCE node
I20260809 06:31:16.110685 10341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:16.112023 10341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:16.113258 10341 hybrid_clock.cc:648] HybridClock initialized: now 1786257076113178 us; error 54 us; skew 500 ppm
I20260809 06:31:16.113579 10341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:16 dist-test-slave-mjjr krb5kdc[10321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:16.120471 10341 init.cc:377] Logged in from keytab as kudu/127.8.131.62@KRBTEST.COM (short username kudu)
I20260809 06:31:16.121914 10341 webserver.cc:460] Webserver started at http://127.8.131.62:44747/ using document root <none> and password file <none>
I20260809 06:31:16.122608 10341 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:16.122738 10341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:16.123066 10341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:16.126036 10341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/data/instance:
uuid: "a91b46199b404b77979806d4b5787226"
format_stamp: "Formatted at 2026-08-09 06:31:16 on dist-test-slave-mjjr"
server_key: "4216eedbe484470b6f89810ec4ad4c14"
server_key_iv: "9acb44fc07370f76168ed7a170dddf5a"
server_key_version: "encryptionkey@0"
I20260809 06:31:16.126848 10341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/wal/instance:
uuid: "a91b46199b404b77979806d4b5787226"
format_stamp: "Formatted at 2026-08-09 06:31:16 on dist-test-slave-mjjr"
server_key: "4216eedbe484470b6f89810ec4ad4c14"
server_key_iv: "9acb44fc07370f76168ed7a170dddf5a"
server_key_version: "encryptionkey@0"
I20260809 06:31:16.132009 10341 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 06:31:16.135321 10366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:16.136591 10341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 06:31:16.136770 10341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/wal
uuid: "a91b46199b404b77979806d4b5787226"
format_stamp: "Formatted at 2026-08-09 06:31:16 on dist-test-slave-mjjr"
server_key: "4216eedbe484470b6f89810ec4ad4c14"
server_key_iv: "9acb44fc07370f76168ed7a170dddf5a"
server_key_version: "encryptionkey@0"
I20260809 06:31:16.136977 10341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:16.160370 10341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:16.164283 10341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:16.164558 10341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:16.182962 10341 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.62:40167
I20260809 06:31:16.182996 10442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.62:40167 every 8 connection(s)
I20260809 06:31:16.184239 10341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/data/info.pb
I20260809 06:31:16.187844 10443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:31:16.194161 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 10341
I20260809 06:31:16.194375 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-0/wal/instance
I20260809 06:31:16.194833 8716 external_mini_cluster.cc:1428] Setting key 683cc4f1ceae6d2145a3ab24ee87663e
I20260809 06:31:16.194149 10443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:16 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:31:16.220180 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.197341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.8.131.62:40167)
0809 06:31:16.199405 (+ 2064us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:16.199412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:16.199454 (+ 42us) server_negotiation.cc:407] Connection header received
0809 06:31:16.201698 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:16.201703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:16.201938 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:16.202096 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:16.203787 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.204998 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.207185 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.208050 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.208215 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:16.212221 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:16.212234 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:16.212251 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:16.212299 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:16.215359 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:16.216356 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:16.216363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:16.216365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:16.216514 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:16.217033 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:16.217039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:16.217042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:16.217666 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:16.217863 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:16.218706 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 06:31:16.219138 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":203,"server-negotiator.queue_time_us":1867,"spinlock_wait_cycles":37120,"thread_start_us":1642,"threads_started":1}
I20260809 06:31:16.220222 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.197341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40167 (local address 127.0.0.1:49256)
0809 06:31:16.198376 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.198428 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:16.201344 (+ 2916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:16.202177 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:16.202186 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:16.203026 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:16.203552 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:16.203579 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.205160 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.205167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:16.207023 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:16.207041 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.208290 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.208300 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:16.208533 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:16.209268 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:16.209290 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:16.211977 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:16.215503 (+ 3526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:16.215517 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:16.215530 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:16.216227 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:16.216661 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:16.216669 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:16.216675 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:16.216894 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:16.217833 (+ 939us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:16.217844 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:16.218145 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 06:31:16.218461 (+ 316us) client_negotiation.cc:240] Negotiation successful
0809 06:31:16.219023 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":461,"threads_started":1}
I20260809 06:31:16.222985 10443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:31:16.224135 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.223772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32864)
0809 06:31:16.223940 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.224038 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 06:31:16.224546 10367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
I20260809 06:31:16.226090 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.225763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32874)
0809 06:31:16.225912 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.225988 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 06:31:16.226368 10443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:43071: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.131.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab.
Entry for principal kudu/127.8.131.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab.
I20260809 06:31:16.263001 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.61:43071
--webserver_interface=127.8.131.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab
--principal=kudu/127.8.131.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:45871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:40167,127.8.131.61:43071,127.8.131.60:44091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:31:16.275755 10443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } attempt: 1
I20260809 06:31:16.276985 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.276547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32880)
0809 06:31:16.276747 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.276836 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 06:31:16.278076 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.277724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32886)
0809 06:31:16.277881 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.277961 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 06:31:16.278323 10443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:43071: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
I20260809 06:31:16.346556 10443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } attempt: 2
I20260809 06:31:16.347913 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.347351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32894)
0809 06:31:16.347599 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.347747 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 06:31:16.348927 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.348580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32898)
0809 06:31:16.348719 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.348779 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 06:31:16.349154 10443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:43071: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
I20260809 06:31:16.440387 10443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } attempt: 3
I20260809 06:31:16.441687 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.441246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32900)
0809 06:31:16.441443 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.441544 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 06:31:16.442816 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.442493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32902)
0809 06:31:16.442628 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.442691 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 06:31:16.443074 10443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:43071: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
I20260809 06:31:16.586323 10443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } attempt: 4
I20260809 06:31:16.587618 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.587074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32914)
0809 06:31:16.587373 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.587490 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260809 06:31:16.588732 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.588415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32918)
0809 06:31:16.588548 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.588622 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 06:31:16.588994 10443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.61:43071: Network error: Client connection negotiation failed: client connection to 127.8.131.61:43071: connect: Connection refused (error 111)
W20260809 06:31:16.606848 10462 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 06:31:16.607211 10462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:16.607331 10462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:16.607468 10462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 06:31:16.614799 10462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:16.614912 10462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:16.615043 10462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:16.615134 10462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:16.615236 10462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:16.624873 10462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:45871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:40167,127.8.131.61:43071,127.8.131.60:44091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.61.keytab
--principal=kudu/127.8.131.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.61:43071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.131.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:16.626107 10462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:16.627487 10462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:16.639982 10471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:16.640018 10475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:16.640057 10473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:16.640858 10462 server_base.cc:1034] running on GCE node
I20260809 06:31:16.641645 10462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:16.642942 10462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:16.644181 10462 hybrid_clock.cc:648] HybridClock initialized: now 1786257076644097 us; error 53 us; skew 500 ppm
I20260809 06:31:16.644500 10462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:16 dist-test-slave-mjjr krb5kdc[10321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:16.647607 10462 init.cc:377] Logged in from keytab as kudu/127.8.131.61@KRBTEST.COM (short username kudu)
I20260809 06:31:16.648969 10462 webserver.cc:460] Webserver started at http://127.8.131.61:40237/ using document root <none> and password file <none>
I20260809 06:31:16.649626 10462 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:16.649753 10462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:16.650063 10462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:16.652920 10462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/data/instance:
uuid: "91248af6766a4c4e87f91bef74fd884b"
format_stamp: "Formatted at 2026-08-09 06:31:16 on dist-test-slave-mjjr"
server_key: "694e03b29873796c67ef16b56e807efd"
server_key_iv: "01e4255391d5604dab5bbfef04289d95"
server_key_version: "encryptionkey@0"
I20260809 06:31:16.653692 10462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/wal/instance:
uuid: "91248af6766a4c4e87f91bef74fd884b"
format_stamp: "Formatted at 2026-08-09 06:31:16 on dist-test-slave-mjjr"
server_key: "694e03b29873796c67ef16b56e807efd"
server_key_iv: "01e4255391d5604dab5bbfef04289d95"
server_key_version: "encryptionkey@0"
I20260809 06:31:16.658468 10462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260809 06:31:16.661762 10483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:16.662932 10462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 06:31:16.663116 10462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/wal
uuid: "91248af6766a4c4e87f91bef74fd884b"
format_stamp: "Formatted at 2026-08-09 06:31:16 on dist-test-slave-mjjr"
server_key: "694e03b29873796c67ef16b56e807efd"
server_key_iv: "01e4255391d5604dab5bbfef04289d95"
server_key_version: "encryptionkey@0"
I20260809 06:31:16.663318 10462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:16.682446 10462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:16.686620 10462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:16.686897 10462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:16.704977 10462 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.61:43071
I20260809 06:31:16.705010 10563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.61:43071 every 8 connection(s)
I20260809 06:31:16.706295 10462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/data/info.pb
I20260809 06:31:16.709913 10564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:31:16.712343 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 10462
I20260809 06:31:16.712544 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-1/wal/instance
I20260809 06:31:16.713124 8716 external_mini_cluster.cc:1428] Setting key 43642998b25953464dc53c9f44aa54d7
I20260809 06:31:16.719074 10564 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:16 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:31:16.743384 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.722462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.8.131.62:40167)
0809 06:31:16.723015 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:16.723022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:16.723981 (+ 959us) server_negotiation.cc:407] Connection header received
0809 06:31:16.726798 (+ 2817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:16.726802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:16.726869 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:16.726966 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:16.728823 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.730064 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.731582 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.732385 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.732547 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:16.735988 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:16.735997 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:16.736004 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:16.736055 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:16.739865 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:16.740631 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:16.740639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:16.740645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:16.740795 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:16.741306 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:16.741314 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:16.741320 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:16.741792 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:16.741949 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:16.742625 (+ 676us) server_negotiation.cc:299] Negotiation successful
0809 06:31:16.742886 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":281,"threads_started":1}
I20260809 06:31:16.743989 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.722925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40167 (local address 127.0.0.1:49272)
0809 06:31:16.723798 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.723853 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:16.726523 (+ 2670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:16.727218 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:16.727231 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:16.728032 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:16.728595 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:16.728611 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.730230 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.730234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:16.731413 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:16.731425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.732606 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.732611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:16.732805 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:16.733511 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:16.733533 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:16.735747 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:16.740020 (+ 4273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:16.740026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:16.740053 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:16.740507 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:16.740922 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:16.740927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:16.740930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:16.741181 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:16.741916 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:16.741921 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:16.742243 (+ 322us) client_negotiation.cc:769] Sending connection context
0809 06:31:16.742467 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 06:31:16.742836 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":321,"threads_started":1}
I20260809 06:31:16.745801 10564 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.131.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.131.60@KRBTEST.COM" created.
Aug 09 06:31:16 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:31:16.760637 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.746582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32930)
0809 06:31:16.746764 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.746785 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:16.746924 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:16.747958 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:16.747965 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:16.748424 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:16.748643 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:16.748655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.750055 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.750059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:16.750881 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:16.750891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.751885 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.751892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:16.752068 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:16.752652 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:16.752681 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:16.755118 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:16.758185 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:16.758192 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:16.758196 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:16.758559 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:16.758925 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:16.758931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:16.758935 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:16.759066 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:16.759656 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:16.759664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:16.759795 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 06:31:16.759956 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 06:31:16.760217 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 06:31:16.760692 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.746782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32930 (local address 127.8.131.61:43071)
0809 06:31:16.747492 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:16.747497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:16.747521 (+ 24us) server_negotiation.cc:407] Connection header received
0809 06:31:16.747561 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:16.747566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:16.747778 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:16.747895 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:16.748808 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.749919 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.750999 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.751709 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.751804 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:16.755317 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:16.755327 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:16.755339 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:16.755367 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:16.758063 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:16.758688 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:16.758698 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:16.758702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:16.758813 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:16.759166 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:16.759173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:16.759177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:16.759503 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:16.759733 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:16.760100 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 06:31:16.760310 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":433,"threads_started":1}
I20260809 06:31:16.762990 10564 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:31:16.764052 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.763573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43696)
0809 06:31:16.763788 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.763903 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 06:31:16.764297 10489 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
I20260809 06:31:16.765810 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.765501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43708)
0809 06:31:16.765633 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.765711 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 06:31:16.766062 10564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:44091: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.131.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab.
Entry for principal kudu/127.8.131.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab.
I20260809 06:31:16.788404 8716 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
/tmp/dist-test-taskn2CWnW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.60:44091
--webserver_interface=127.8.131.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab
--principal=kudu/127.8.131.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.131.20:45871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.131.62:40167,127.8.131.61:43071,127.8.131.60:44091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 06:31:16.815373 10564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } attempt: 1
I20260809 06:31:16.816730 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.816247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43712)
0809 06:31:16.816458 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.816569 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 06:31:16.817808 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.817442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43726)
0809 06:31:16.817619 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.817698 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 06:31:16.818069 10564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:44091: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
I20260809 06:31:16.886351 10564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } attempt: 2
I20260809 06:31:16.887550 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.887103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43738)
0809 06:31:16.887295 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.887389 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 06:31:16.888263 10443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } attempt: 5
I20260809 06:31:16.888624 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.888218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43750)
0809 06:31:16.888363 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.888464 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 06:31:16.888947 10564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:44091: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Aug 09 06:31:16 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:31:16.901202 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.889143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.8.131.61:43071)
0809 06:31:16.889325 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:16.889331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:16.889352 (+ 21us) server_negotiation.cc:407] Connection header received
0809 06:31:16.889466 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:16.889471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:16.889551 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:16.889667 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:16.890467 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.891161 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.892402 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.892837 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.892967 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:16.895988 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:16.895993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:16.895996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:16.896026 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:16.898570 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:16.899198 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:16.899205 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:16.899209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:16.899349 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:16.899768 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:16.899773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:16.899776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:16.900051 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:16.900208 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:16.900648 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 06:31:16.900858 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:31:16.901197 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.888913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32936)
0809 06:31:16.889182 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.889216 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:16.889350 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:16.889693 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:16.889697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:16.890139 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:16.890318 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:16.890327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.891296 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.891302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:16.892243 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:16.892255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:16.892986 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:16.892992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:16.893190 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:16.893831 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:16.893848 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:16.895813 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:16.898717 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:16.898722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:16.898726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:16.899086 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:16.899480 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:16.899485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:16.899489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:16.899625 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:16.900169 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:16.900174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:16.900349 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 06:31:16.900533 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 06:31:16.900821 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260809 06:31:16.902405 10443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
I20260809 06:31:16.903707 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.903212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43760)
0809 06:31:16.903400 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.903495 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 06:31:16.904635 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.904313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43768)
0809 06:31:16.904446 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.904524 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 06:31:16.904904 10443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:44091: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
I20260809 06:31:16.956135 10443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } attempt: 1
I20260809 06:31:16.957446 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.956970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43784)
0809 06:31:16.957173 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.957270 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 06:31:16.958405 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.958123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43788)
0809 06:31:16.958243 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.958307 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 06:31:16.958647 10443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:44091: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
I20260809 06:31:16.980170 10564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } attempt: 3
I20260809 06:31:16.981348 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.980860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43796)
0809 06:31:16.981086 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.981190 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 06:31:16.982403 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:16.982006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43810)
0809 06:31:16.982218 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:16.982296 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260809 06:31:16.982689 10564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:44091: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
I20260809 06:31:17.026903 10443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } attempt: 2
I20260809 06:31:17.028203 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.027726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43824)
0809 06:31:17.027937 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.028042 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 06:31:17.029203 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.028846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43834)
0809 06:31:17.029000 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.029087 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 06:31:17.029420 10443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:44091: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
I20260809 06:31:17.125927 10564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } attempt: 4
I20260809 06:31:17.127174 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.126717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43842)
0809 06:31:17.126933 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.127022 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 06:31:17.128122 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.127813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43844)
0809 06:31:17.127963 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.128025 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 06:31:17.128306 10564 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:44091: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
W20260809 06:31:17.133841 10574 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 06:31:17.134195 10574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 06:31:17.134320 10574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 06:31:17.134465 10574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 06:31:17.135609 10443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } attempt: 3
I20260809 06:31:17.136615 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.136195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43846)
0809 06:31:17.136371 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.136462 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 06:31:17.137550 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.137196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43852)
0809 06:31:17.137390 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.137452 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260809 06:31:17.137737 10443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.131.60:44091: Network error: Client connection negotiation failed: client connection to 127.8.131.60:44091: connect: Connection refused (error 111)
W20260809 06:31:17.142171 10574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 06:31:17.142287 10574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 06:31:17.142395 10574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 06:31:17.142490 10574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 06:31:17.142598 10574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 06:31:17.152704 10574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.131.20:45871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.131.62:40167,127.8.131.61:43071,127.8.131.60:44091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/krb5kdc/kudu_127.8.131.60.keytab
--principal=kudu/127.8.131.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.131.60:44091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.131.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 05:36:50 UTC on ccycloud.quasar-ooxuuz.root.comops.site
build id 4575857
ASAN enabled
I20260809 06:31:17.153998 10574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 06:31:17.155295 10574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 06:31:17.167232 10582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:17.167279 10589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 06:31:17.167274 10583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 06:31:17.168161 10574 server_base.cc:1034] running on GCE node
I20260809 06:31:17.168849 10574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 06:31:17.170127 10574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 06:31:17.171339 10574 hybrid_clock.cc:648] HybridClock initialized: now 1786257077171276 us; error 40 us; skew 500 ppm
I20260809 06:31:17.171627 10574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 06:31:17 dist-test-slave-mjjr krb5kdc[10321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257077, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 06:31:17.174626 10574 init.cc:377] Logged in from keytab as kudu/127.8.131.60@KRBTEST.COM (short username kudu)
I20260809 06:31:17.175981 10574 webserver.cc:460] Webserver started at http://127.8.131.60:43985/ using document root <none> and password file <none>
I20260809 06:31:17.176628 10574 fs_manager.cc:359] Metadata directory not provided
I20260809 06:31:17.176740 10574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 06:31:17.177084 10574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 06:31:17.180056 10574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/data/instance:
uuid: "33a6755cfcb64153b71f6807c4c753c0"
format_stamp: "Formatted at 2026-08-09 06:31:17 on dist-test-slave-mjjr"
server_key: "be30e55a827e35e336c2e69364210441"
server_key_iv: "c61fc4b2d8cb1ab811b0039c73b7feef"
server_key_version: "encryptionkey@0"
I20260809 06:31:17.180889 10574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/wal/instance:
uuid: "33a6755cfcb64153b71f6807c4c753c0"
format_stamp: "Formatted at 2026-08-09 06:31:17 on dist-test-slave-mjjr"
server_key: "be30e55a827e35e336c2e69364210441"
server_key_iv: "c61fc4b2d8cb1ab811b0039c73b7feef"
server_key_version: "encryptionkey@0"
I20260809 06:31:17.185967 10574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 06:31:17.189433 10598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 06:31:17.190801 10574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 06:31:17.190981 10574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/data,/tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/wal
uuid: "33a6755cfcb64153b71f6807c4c753c0"
format_stamp: "Formatted at 2026-08-09 06:31:17 on dist-test-slave-mjjr"
server_key: "be30e55a827e35e336c2e69364210441"
server_key_iv: "c61fc4b2d8cb1ab811b0039c73b7feef"
server_key_version: "encryptionkey@0"
I20260809 06:31:17.191183 10574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 06:31:17.203953 10574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 06:31:17.207722 10574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 06:31:17.207983 10574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 06:31:17.225813 10574 rpc_server.cc:307] RPC server started. Bound to: 127.8.131.60:44091
I20260809 06:31:17.225831 10694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.131.60:44091 every 8 connection(s)
I20260809 06:31:17.227164 10574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/data/info.pb
I20260809 06:31:17.230527 8716 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu as pid 10574
I20260809 06:31:17.230746 8716 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn2CWnW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786256699676692-8716-0/minicluster-data/master-2/wal/instance
I20260809 06:31:17.231030 10695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 06:31:17.231343 8716 external_mini_cluster.cc:1428] Setting key 941acf70a8541fc91ce8ccb94e0b2e6b
I20260809 06:31:17.240885 10695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:17 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
Aug 09 06:31:17 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257077, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
I20260809 06:31:17.254717 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.233175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.8.131.60:44091)
0809 06:31:17.233987 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.233998 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.234041 (+ 43us) server_negotiation.cc:407] Connection header received
0809 06:31:17.235906 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.235932 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.236298 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.236613 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.238523 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.239887 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.241730 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.242421 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.242541 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.247225 (+ 4684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.247232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.247244 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.247279 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.250519 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.251179 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.251209 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.251214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.251359 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.251832 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.251840 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.251845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.252574 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.252870 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.253279 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.253693 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":366,"threads_started":1}
I20260809 06:31:17.258473 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.243707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.62:40167 (local address 127.0.0.1:49274)
0809 06:31:17.244754 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.244794 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:17.244937 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:17.245370 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:17.245374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:17.245949 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:17.246186 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.246194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.247170 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.247174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.248302 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.248314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.249097 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.249105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.249266 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.249850 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:17.249873 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:17.252287 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:17.255603 (+ 3316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.255615 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.255622 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.256132 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.256516 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.256521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.256524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.256650 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.257317 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:17.257324 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:17.257539 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 06:31:17.257758 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 06:31:17.258042 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":652,"threads_started":1}
I20260809 06:31:17.258512 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.243726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49274 (local address 127.8.131.62:40167)
0809 06:31:17.244332 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.244340 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.244894 (+ 554us) server_negotiation.cc:407] Connection header received
0809 06:31:17.245089 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.245096 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.245198 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.245306 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.246334 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.247009 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.248487 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.248891 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.249040 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.252506 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.252515 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.252519 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.252556 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.255479 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.256254 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.256260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.256263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.256393 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.256797 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.256803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.256806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.257173 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.257335 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.257869 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.258086 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":355,"threads_started":1}
I20260809 06:31:17.259793 10695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:17 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.62@KRBTEST.COM
Aug 09 06:31:17 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257077, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:31:17.273634 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.257319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.8.131.62:40167)
0809 06:31:17.257903 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.257907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.257927 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:31:17.257963 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.257966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.258014 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.258120 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.259518 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.260602 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.262088 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.262717 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.262811 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.267574 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.267582 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.267587 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.267624 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.270919 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.271539 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.271543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.271545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.271715 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.272095 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.272099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.272102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.272445 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.272663 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.273023 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.273254 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":268,"threads_started":1}
I20260809 06:31:17.274080 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.260377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.61:43071 (local address 127.0.0.1:32940)
0809 06:31:17.260524 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.260543 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:17.260627 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:17.260957 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:17.260962 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:17.261373 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:17.261550 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.261560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.262687 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.262691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.264106 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.264122 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.264794 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.264801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.265006 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.265479 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:17.265502 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:17.267879 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:17.271460 (+ 3581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.271468 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.271474 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.271961 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.272377 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.272380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.272382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.272559 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.273191 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:17.273198 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:17.273348 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 06:31:17.273505 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 06:31:17.273756 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 06:31:17.274225 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.260473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32940 (local address 127.8.131.61:43071)
0809 06:31:17.260693 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.260699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.260718 (+ 19us) server_negotiation.cc:407] Connection header received
0809 06:31:17.260778 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.260781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.260839 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.260910 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.261694 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.262542 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.264245 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.264636 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.264735 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.268038 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.268046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.268050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.268093 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.271334 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.272086 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.272091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.272094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.272264 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.272710 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.272715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.272718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.273046 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.273204 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.273630 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.273835 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:31:17.275018 10695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 06:31:17 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257077, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.60@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
Aug 09 06:31:17 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.131.61@KRBTEST.COM
I20260809 06:31:17.288249 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.275625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43866)
0809 06:31:17.275809 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.275826 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:17.275926 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:17.276254 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:17.276257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:17.276574 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:17.276693 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.276700 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.277826 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.277831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.278872 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.278883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.279793 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.279799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.280014 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.280642 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:17.280658 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:17.283028 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:17.285960 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.285966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.285970 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.286310 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.286663 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.286669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.286673 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.286803 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.287275 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:17.287279 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:17.287412 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 06:31:17.287553 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 06:31:17.287862 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 06:31:17.288432 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.275692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.8.131.60:44091)
0809 06:31:17.275827 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.275832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.275872 (+ 40us) server_negotiation.cc:407] Connection header received
0809 06:31:17.276041 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.276046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.276116 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.276214 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.276829 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.277702 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.279002 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.279635 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.279766 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.283182 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.283186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.283190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.283216 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.285830 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.286429 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.286434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.286441 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.286563 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.286902 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.286909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.286912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.287156 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.287294 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.287671 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.288032 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":130,"server-negotiator.queue_time_us":44}
I20260809 06:31:17.289973 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.275534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32956 (local address 127.8.131.61:43071)
0809 06:31:17.275724 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.275730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.275747 (+ 17us) server_negotiation.cc:407] Connection header received
0809 06:31:17.275791 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.275795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.275869 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.275955 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.277346 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.278238 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.279599 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.280290 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.280411 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.284850 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.284855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.284859 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.284896 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.287350 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.288067 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.288074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.288077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.288216 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.288625 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.288630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.288634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.288949 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.289203 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.289469 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.289680 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 06:31:17.293892 10695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0: Bootstrap starting.
I20260809 06:31:17.294368 8716 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 06:31:17.297274 10695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:17.298261 10695 log.cc:824] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0: Log is configured to *not* fsync() on all Append() calls
I20260809 06:31:17.296717 10406 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a91b46199b404b77979806d4b5787226"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49278
I20260809 06:31:17.301033 10695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0: No bootstrap required, opened a new log
I20260809 06:31:17.308168 10695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } }
I20260809 06:31:17.308491 10695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a6755cfcb64153b71f6807c4c753c0, State: Initialized, Role: FOLLOWER
I20260809 06:31:17.308897 10695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } }
I20260809 06:31:17.309804 10707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.310178 10707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:17.310644 10695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:31:17.314965 10443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } attempt: 4
W20260809 06:31:17.324301 10726 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 06:31:17.324461 10726 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 06:31:17 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.62@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
I20260809 06:31:17.317988 10574 master_runner.cc:186] Error getting master registration for 127.8.131.62:40167: OK, instance_id { permanent_uuid: "a91b46199b404b77979806d4b5787226" instance_seqno: 1786257076177551 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 06:31:17.329988 10574 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 06:31:17.331897 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.315675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43870)
0809 06:31:17.315889 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.315910 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:17.316048 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:17.316480 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:17.316487 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:17.316981 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:17.317216 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.317229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.318782 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.318789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.320335 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.320353 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.321430 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.321439 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.321673 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.322427 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:17.322451 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:17.325391 (+ 2940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:17.329262 (+ 3871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.329268 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.329272 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.329713 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.330145 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.330150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.330154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.330292 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.330880 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:17.330887 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:17.331040 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 06:31:17.331174 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 06:31:17.331489 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 06:31:17.331897 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.315895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.8.131.60:44091)
0809 06:31:17.316087 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.316092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.316136 (+ 44us) server_negotiation.cc:407] Connection header received
0809 06:31:17.316210 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.316214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.316298 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.316420 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.317383 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.318582 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.320523 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.321254 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.321384 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.325568 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.325573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.325582 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.325615 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.329131 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.329840 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.329845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.329849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.329996 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.330417 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.330422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.330425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.330746 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.330912 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.331311 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.331509 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:31:17.337700 10443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226: Bootstrap starting.
I20260809 06:31:17.341923 10443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:17.343255 10443 log.cc:824] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226: Log is configured to *not* fsync() on all Append() calls
I20260809 06:31:17.346629 10443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226: No bootstrap required, opened a new log
I20260809 06:31:17.354187 10443 raft_consensus.cc:348] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } }
I20260809 06:31:17.354595 10443 raft_consensus.cc:729] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a91b46199b404b77979806d4b5787226, State: Initialized, Role: FOLLOWER
I20260809 06:31:17.355086 10443 consensus_queue.cc:260] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } }
I20260809 06:31:17.356135 10729 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.356555 10729 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:17.357402 10443 sys_catalog.cc:564] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:31:17.357487 10406 raft_consensus.cc:482] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 06:31:17.357739 10406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:17.360586 10406 raft_consensus.cc:504] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } }
I20260809 06:31:17.362933 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a91b46199b404b77979806d4b5787226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "33a6755cfcb64153b71f6807c4c753c0"
I20260809 06:31:17.363449 10655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:17.363775 10406 leader_election.cc:288] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [CANDIDATE]: Term 1 election: Requested vote from peers 91248af6766a4c4e87f91bef74fd884b (127.8.131.61:43071), 33a6755cfcb64153b71f6807c4c753c0 (127.8.131.60:44091)
I20260809 06:31:17.368474 10655 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a91b46199b404b77979806d4b5787226 in term 1.
I20260809 06:31:17.369254 10369 leader_election.cc:302] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33a6755cfcb64153b71f6807c4c753c0, a91b46199b404b77979806d4b5787226; no voters:
I20260809 06:31:17.369669 10729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 06:31:17.378940 10729 raft_consensus.cc:686] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [term 1 LEADER]: Becoming Leader. State: Replica: a91b46199b404b77979806d4b5787226, State: Running, Role: LEADER
I20260809 06:31:17.380096 10729 consensus_queue.cc:237] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } }
I20260809 06:31:17.408330 10742 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a91b46199b404b77979806d4b5787226. Latest consensus state: current_term: 1 leader_uuid: "a91b46199b404b77979806d4b5787226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.408794 10742 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [sys.catalog]: This master's current role is: LEADER
W20260809 06:31:17.417308 10373 tablet.cc:2369] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:31:17.420284 10760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 06:31:17.427241 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.392813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.8.131.61:43071)
0809 06:31:17.393057 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.393066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.405890 (+ 12824us) server_negotiation.cc:407] Connection header received
0809 06:31:17.406150 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.406157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.406248 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.406369 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.408739 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.410148 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.412020 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.412987 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.413151 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.418387 (+ 5236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.418393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.418399 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.418457 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.422220 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.423880 (+ 1660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.423886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.423892 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.424067 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.424920 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.424927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.424933 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.425309 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.425569 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.426636 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.426878 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 06:31:17.427474 10564 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } attempt: 5
I20260809 06:31:17.430272 10761 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 06:31:17.433610 10760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 06:31:17.436470 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.397641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49280 (local address 127.8.131.62:40167)
0809 06:31:17.397885 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.397893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.406893 (+ 9000us) server_negotiation.cc:407] Connection header received
0809 06:31:17.416253 (+ 9360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.416259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.416345 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.416424 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.420021 (+ 3597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.421193 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.423206 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.423917 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.424033 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.428249 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.428262 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.428271 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.428315 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.432062 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.432894 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.432899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.432902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.433102 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.433573 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.433579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.433582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.434025 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.434292 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.435780 (+ 1488us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.436067 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 06:31:17 dist-test-slave-mjjr krb5kdc[10321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786257076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.131.61@KRBTEST.COM for kudu/127.8.131.60@KRBTEST.COM
I20260809 06:31:17.434676 10341 master_runner.cc:186] Error getting master registration for 127.8.131.61:43071: OK, instance_id { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" instance_seqno: 1786257076699894 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 06:31:17.445247 10341 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 06:31:17.446907 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.428000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.131.60:44091 (local address 127.0.0.1:43876)
0809 06:31:17.428408 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 06:31:17.428432 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 06:31:17.428571 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 06:31:17.429957 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 06:31:17.429965 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 06:31:17.430732 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0809 06:31:17.430937 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.430951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.433354 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.433361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.434778 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 06:31:17.434791 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.435899 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.435906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 06:31:17.436121 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.436716 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 06:31:17.436744 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 06:31:17.439478 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 06:31:17.443578 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.443583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.443587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.444133 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.444662 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 06:31:17.444667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 06:31:17.444670 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 06:31:17.444929 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 06:31:17.445872 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 06:31:17.445879 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 06:31:17.446052 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 06:31:17.446207 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 06:31:17.446512 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255}
I20260809 06:31:17.447475 10766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.428834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43876 (local address 127.8.131.60:44091)
0809 06:31:17.429616 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.429622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.429645 (+ 23us) server_negotiation.cc:407] Connection header received
0809 06:31:17.429698 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.429704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.429777 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.431096 (+ 1319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.431348 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.433189 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.434968 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.435711 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.438156 (+ 2445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.439721 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.439730 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.439735 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.439776 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.443454 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.444306 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.444312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.444316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.444520 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.445183 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.445193 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.445196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.445680 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.445852 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.446372 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.446758 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":643,"thread_start_us":1624,"threads_started":1}
I20260809 06:31:17.453120 10564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b: Bootstrap starting.
I20260809 06:31:17.457158 10564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b: Neither blocks nor log segments found. Creating new log.
I20260809 06:31:17.458550 10564 log.cc:824] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b: Log is configured to *not* fsync() on all Append() calls
W20260809 06:31:17.459959 10612 tablet.cc:2369] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 06:31:17.462481 10564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b: No bootstrap required, opened a new log
I20260809 06:31:17.462790 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.413972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.8.131.60:44091)
0809 06:31:17.414139 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.414144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.422873 (+ 8729us) server_negotiation.cc:407] Connection header received
0809 06:31:17.423168 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.423178 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.423284 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.423370 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.427168 (+ 3798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.428075 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.444684 (+ 16609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.445427 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.445530 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.455821 (+ 10291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 06:31:17.455828 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 06:31:17.455833 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 06:31:17.455874 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 06:31:17.459517 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.460368 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.460376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.460380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.460546 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 06:31:17.461007 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 06:31:17.461012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 06:31:17.461016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 06:31:17.461353 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 06:31:17.461617 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.462146 (+ 529us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.462379 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":43}
I20260809 06:31:17.473762 10655 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [term 1 FOLLOWER]: Refusing update from remote peer a91b46199b404b77979806d4b5787226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:31:17.474771 10742 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:31:17.474665 10564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } }
I20260809 06:31:17.475826 10564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91248af6766a4c4e87f91bef74fd884b, State: Initialized, Role: FOLLOWER
I20260809 06:31:17.476328 10564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } }
I20260809 06:31:17.477656 10773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.478461 10773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:17.479321 10564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 06:31:17.479362 10523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a91b46199b404b77979806d4b5787226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "91248af6766a4c4e87f91bef74fd884b"
I20260809 06:31:17.479808 10523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [term 0 FOLLOWER]: Advancing to term 1
I20260809 06:31:17.484153 10523 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a91b46199b404b77979806d4b5787226 in term 1.
I20260809 06:31:17.485340 10522 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [term 1 FOLLOWER]: Refusing update from remote peer a91b46199b404b77979806d4b5787226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 06:31:17.486279 10742 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 06:31:17.503461 10773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a91b46199b404b77979806d4b5787226. Latest consensus state: current_term: 1 leader_uuid: "a91b46199b404b77979806d4b5787226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.503819 10773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:17.504737 10707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a91b46199b404b77979806d4b5787226. Latest consensus state: current_term: 1 leader_uuid: "a91b46199b404b77979806d4b5787226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.505086 10707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:17.506394 10768 mvcc.cc:205] Tried to move back new op lower bound from 7316508989307416576 to 7316508988960616448. Current Snapshot: MvccSnapshot[applied={T|T < 7316508989307416576}]
I20260809 06:31:17.508828 10742 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a91b46199b404b77979806d4b5787226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.509200 10742 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:17.509497 10742 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a91b46199b404b77979806d4b5787226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.509754 10742 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226 [sys.catalog]: This master's current role is: LEADER
I20260809 06:31:17.509871 10760 catalog_manager.cc:1347] Generated new cluster ID: 1df416d04ff544feae531165c7cb6b27
I20260809 06:31:17.509968 10760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 06:31:17.531108 10773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a91b46199b404b77979806d4b5787226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.531497 10773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:17.541411 10707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a91b46199b404b77979806d4b5787226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91b46199b404b77979806d4b5787226" member_type: VOTER last_known_addr { host: "127.8.131.62" port: 40167 } } peers { permanent_uuid: "91248af6766a4c4e87f91bef74fd884b" member_type: VOTER last_known_addr { host: "127.8.131.61" port: 43071 } } peers { permanent_uuid: "33a6755cfcb64153b71f6807c4c753c0" member_type: VOTER last_known_addr { host: "127.8.131.60" port: 44091 } } }
I20260809 06:31:17.541774 10707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 06:31:17.558945 10760 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 06:31:17.560889 10760 catalog_manager.cc:1504] Loading token signing keys...
I20260809 06:31:17.574991 10797 catalog_manager.cc:1259] Loaded cluster ID: 1df416d04ff544feae531165c7cb6b27
I20260809 06:31:17.576735 10797 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b: loading cluster ID for follower catalog manager: success
I20260809 06:31:17.580474 10760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a91b46199b404b77979806d4b5787226: Generated new TSK 0
I20260809 06:31:17.581434 10760 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 06:31:17.591257 10797 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b: acquiring CA information for follower catalog manager: success
I20260809 06:31:17.593787 10797 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91248af6766a4c4e87f91bef74fd884b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 06:31:17.721771 10705 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:17.702098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.8.131.62:40167)
0809 06:31:17.702309 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.702317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.719600 (+ 17283us) server_negotiation.cc:407] Connection header received
0809 06:31:17.720095 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.720101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.720187 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.720258 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.721480 (+ 1222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49294: BlockingRecv error: recv got EOF from 127.0.0.1:49294 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 06:31:17.724011 10571 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:17.704093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32970 (local address 127.8.131.61:43071)
0809 06:31:17.704310 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.704317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.722369 (+ 18052us) server_negotiation.cc:407] Connection header received
0809 06:31:17.722641 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.722648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.722749 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.722820 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.723777 (+ 957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32970: BlockingRecv error: recv got EOF from 127.0.0.1:32970 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 06:31:17.727878 10699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:17.725367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43882 (local address 127.8.131.60:44091)
0809 06:31:17.725557 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.725564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.726092 (+ 528us) server_negotiation.cc:407] Connection header received
0809 06:31:17.726326 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.726332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.726430 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.726519 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.727593 (+ 1074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43882: BlockingRecv error: recv got EOF from 127.0.0.1:43882 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 06:31:17.780356 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.760209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32978 (local address 127.8.131.61:43071)
0809 06:31:17.760387 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.760394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.771228 (+ 10834us) server_negotiation.cc:407] Connection header received
0809 06:31:17.771894 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.771901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.772116 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.772251 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:17.773014 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.775197 (+ 2183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.777250 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.777924 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.778028 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.778786 (+ 758us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:17.779476 (+ 690us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:17.779601 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.779884 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.779986 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 06:31:17.823302 10699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:17.813132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.8.131.60:44091)
0809 06:31:17.813353 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.813359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.821109 (+ 7750us) server_negotiation.cc:407] Connection header received
0809 06:31:17.821644 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.821651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.821778 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.821863 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.823040 (+ 1177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43896: BlockingRecv error: recv got EOF from 127.0.0.1:43896 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 06:31:17.894754 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.870777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32992 (local address 127.8.131.61:43071)
0809 06:31:17.871000 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.871006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.886235 (+ 15229us) server_negotiation.cc:407] Connection header received
0809 06:31:17.886799 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.886806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.886944 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.887022 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:17.887759 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.890230 (+ 2471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.892370 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.893057 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.893156 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.893819 (+ 663us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:17.893986 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:17.894067 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.894316 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.894401 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260809 06:31:17.938361 10699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:17.928244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.8.131.60:44091)
0809 06:31:17.928461 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.928468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.936318 (+ 7850us) server_negotiation.cc:407] Connection header received
0809 06:31:17.936806 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.936816 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.936932 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.937011 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:17.938085 (+ 1074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43900: BlockingRecv error: recv got EOF from 127.0.0.1:43900 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 06:31:17.988095 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:17.971859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33008 (local address 127.8.131.61:43071)
0809 06:31:17.972056 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:17.972063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:17.980454 (+ 8391us) server_negotiation.cc:407] Connection header received
0809 06:31:17.981038 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:17.981045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:17.981148 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:17.981234 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:17.982112 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.983287 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.985614 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:17.986383 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:17.986512 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:17.987164 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:17.987314 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:17.987378 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:17.987617 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 06:31:17.987730 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 06:31:18.019279 10699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:18.016707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43912 (local address 127.8.131.60:44091)
0809 06:31:18.016925 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.016934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.017597 (+ 663us) server_negotiation.cc:407] Connection header received
0809 06:31:18.017829 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.017836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.017945 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.018075 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:18.019000 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43912: BlockingRecv error: recv got EOF from 127.0.0.1:43912 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 06:31:18.036482 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.028128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.8.131.61:43071)
0809 06:31:18.028405 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.028411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.029645 (+ 1234us) server_negotiation.cc:407] Connection header received
0809 06:31:18.029872 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.029878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.029982 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.030109 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.030667 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.031952 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.034078 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.034816 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.034947 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.035563 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.035733 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.035824 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.036063 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.036150 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
W20260809 06:31:18.053139 10699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:18.050068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.8.131.60:44091)
0809 06:31:18.050372 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.050379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.051006 (+ 627us) server_negotiation.cc:407] Connection header received
0809 06:31:18.051372 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.051380 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.051505 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.051603 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:18.052850 (+ 1247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43916: BlockingRecv error: recv got EOF from 127.0.0.1:43916 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
I20260809 06:31:18.074101 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.066121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.8.131.61:43071)
0809 06:31:18.066432 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.066438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.067061 (+ 623us) server_negotiation.cc:407] Connection header received
0809 06:31:18.067412 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.067418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.067511 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.067618 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.068260 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.069443 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.071714 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.072511 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.072633 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.073227 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.073390 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.073479 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.073662 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.073746 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
W20260809 06:31:18.085660 10699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:18.083381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43922 (local address 127.8.131.60:44091)
0809 06:31:18.083549 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.083555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.084097 (+ 542us) server_negotiation.cc:407] Connection header received
0809 06:31:18.084250 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.084257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.084365 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.084485 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:18.085382 (+ 897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43922: BlockingRecv error: recv got EOF from 127.0.0.1:43922 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 06:31:18.111024 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.104888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.8.131.61:43071)
0809 06:31:18.105095 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.105103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.105391 (+ 288us) server_negotiation.cc:407] Connection header received
0809 06:31:18.105611 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.105615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.105684 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.105763 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.106339 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.107150 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.109239 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.109678 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.109770 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.110343 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.110457 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.110523 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.110702 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.110772 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 06:31:18.119107 10699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:18.117175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.8.131.60:44091)
0809 06:31:18.117406 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.117415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.117743 (+ 328us) server_negotiation.cc:407] Connection header received
0809 06:31:18.117894 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.117902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.118007 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.118133 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:18.118848 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43930: BlockingRecv error: recv got EOF from 127.0.0.1:43930 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260809 06:31:18.160780 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.154486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33038 (local address 127.8.131.61:43071)
0809 06:31:18.154748 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.154756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.155115 (+ 359us) server_negotiation.cc:407] Connection header received
0809 06:31:18.155300 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.155307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.155429 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.155567 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.155992 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.157133 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.158551 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.159338 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.159496 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.159889 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.160046 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.160170 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.160293 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.160380 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 06:31:18.168692 10699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:18.166827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43936 (local address 127.8.131.60:44091)
0809 06:31:18.167040 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.167046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.167323 (+ 277us) server_negotiation.cc:407] Connection header received
0809 06:31:18.167518 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.167523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.167614 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.167771 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:18.168461 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43936: BlockingRecv error: recv got EOF from 127.0.0.1:43936 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 06:31:18.242007 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.236042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.8.131.61:43071)
0809 06:31:18.236604 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.236608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.236628 (+ 20us) server_negotiation.cc:407] Connection header received
0809 06:31:18.236858 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.236862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.236927 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.237006 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.237502 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.238214 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.240218 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.240666 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.240758 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.241284 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.241393 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.241502 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.241656 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.241736 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 06:31:18.250214 10699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 06:31:18.248373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43952 (local address 127.8.131.60:44091)
0809 06:31:18.248595 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.248603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.248857 (+ 254us) server_negotiation.cc:407] Connection header received
0809 06:31:18.249065 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.249069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.249139 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.249239 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 06:31:18.250050 (+ 811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43952: BlockingRecv error: recv got EOF from 127.0.0.1:43952 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:31:18.326488 10726 catalog_manager.cc:1259] Loaded cluster ID: 1df416d04ff544feae531165c7cb6b27
I20260809 06:31:18.326632 10726 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0: loading cluster ID for follower catalog manager: success
I20260809 06:31:18.329170 10726 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0: acquiring CA information for follower catalog manager: success
I20260809 06:31:18.331007 10726 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33a6755cfcb64153b71f6807c4c753c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 06:31:18.388937 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.382068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.8.131.61:43071)
0809 06:31:18.382327 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.382336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.382767 (+ 431us) server_negotiation.cc:407] Connection header received
0809 06:31:18.383002 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.383006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.383087 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.383189 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.383616 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.384357 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.385642 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.386305 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.387465 (+ 1160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.388017 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.388155 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.388270 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.388461 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.388570 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 06:31:18.404727 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.397354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.8.131.60:44091)
0809 06:31:18.397605 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.397612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.398177 (+ 565us) server_negotiation.cc:407] Connection header received
0809 06:31:18.398392 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.398396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.398623 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.398745 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.399266 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.400178 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.402336 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.402855 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.402987 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.403513 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.404134 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.404314 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.404368 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.404453 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 06:31:18.418103 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.410315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.8.131.61:43071)
0809 06:31:18.410605 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.410614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.411775 (+ 1161us) server_negotiation.cc:407] Connection header received
0809 06:31:18.412118 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.412125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.412224 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.412288 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.413438 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.414487 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.416273 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.416745 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.416865 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.417354 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.417497 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.417594 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.417738 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.417818 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 06:31:18.418896 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.411813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.8.131.60:44091)
0809 06:31:18.411993 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.411998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.412020 (+ 22us) server_negotiation.cc:407] Connection header received
0809 06:31:18.412291 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.412299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.412383 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.412462 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.413451 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.414201 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.416708 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.417388 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.417493 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.418085 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.418190 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.418262 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.418502 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.418598 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 06:31:18.419288 10956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 06:31:18.410147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.8.131.62:40167)
0809 06:31:18.411085 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0809 06:31:18.411094 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 06:31:18.411696 (+ 602us) server_negotiation.cc:407] Connection header received
0809 06:31:18.411825 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 06:31:18.411831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 06:31:18.412188 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 06:31:18.412376 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 06:31:18.413174 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.414367 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.416767 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 06:31:18.417222 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 06:31:18.417364 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 06:31:18.417903 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 06:31:18.418379 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 06:31:18.418491 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 06:31:18.418702 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 06:31:18.418789 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":602,"threads_started":1}
I20260809 06:31:18.423632 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 10341
I20260809 06:31:18.424851 10437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:31:18.535526 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 10341
I20260809 06:31:18.562170 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 10462
I20260809 06:31:18.563289 10558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:31:18.669093 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 10462
I20260809 06:31:18.694726 8716 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu pid 10574
I20260809 06:31:18.695869 10689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 06:31:18.803165 8716 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn2CWnW/build/asan/bin/kudu with pid 10574
2026-08-09T06:31:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4743 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9921 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (379098 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST